CAIQUM-8338: AIQUM services getting crashed after upgrade to 9.18
Issue
- After upgrade of AIQUM to 9.18 version, the ocie service is crashing repeatedly
- Active IQ Unified Manager GUI not accessible.
- Active IQ Unified Manager OVA shows status as 'Not Running' in the maintenance console.
- This is seen in environments monitoring 40-100+ systems.
- In the support bundle there are logs named like:
JVM-LOGS_JAVA-ERROR-<id>.log in a jvm-logs directory - If the errors are not in the support bundle, check the OS specific locations:
- Windows:
<install disk>:\ProgramData\NetApp\OnCommandAppData\ocum\temp\java_error_p.log - OVA/RHEL:
/var/log/ocum/jvm/java_error_%p.log - The errors will look similar to the one below when viewing it with a text editor
~ ??A fatal error has been detected by the Java Runtime Environment:
#
Internal Error (type.cpp:1159), pid=2226698, tid=2239965
Error: ShouldNotReachHere()
#
JRE version: OpenJDK Runtime Environment (Red_Hat-17.0.17.0.10-1) (17.0.17+10) (build 17.0.17+10-LTS)
Java VM: OpenJDK 64-Bit Server VM (Red_Hat-17.0.17.0.10-1) (17.0.17+10-LTS, mixed mode, sharing, tiered, compressed class ptrs, g1 gc, linux-amd64)
Problematic frame:
V [libjvm.so+0xee5156] TypeVect::xmeet(Type const*) const+0x66
#
Core dump will be written. Default location: Core dumps may be processed with "/usr/lib/systemd/systemd-coredump %P %u %g %s %t %c %h %e" (or dumping to //core.2226698)
#
If you would like to submit a bug report, please visit:
https://access.redhat.com/support/cases/??~
ocumserver.log:
[tsaTaskExecutor-0] Running TSA forecast...
[tsaTaskExecutor-0] starting new TSA for cluster...
[tsaTaskExecutor-0] Not enough sample found for object...journalctl.txtincluded in a vApp/Linux support bundle indicates that OCIE was killed withstatus=6/ABRT
systemd[1]: ocie.service: Main process exited, code=killed, status=6/ABRT
systemd[1]: ocie.service: Failed with result 'signal'.
systemd[1]: ocie.service: Consumed Xh XXmin XX.XXXs CPU time.
systemd[1]: ocie.service: Scheduled restart job, restart counter is at 1.
systemd[1]: Stopped ocie.service - LSB: Active IQ Management Server.
systemd[1]: ocie.service: Consumed Xh XXmin XX.XXXs CPU time.
systemd[1]: Starting ocie.service - LSB: Active IQ Management Server...
systemd[1]: Reloading.- Data acquisition can keep failing with the following error in
au.log
ERROR [Force Acq Listener] c.o.s.a.f.m.ServiceManager (ServiceManager.java:491) - Data Source Events Manager reported a fatal error: Failed to communicate with server. Acquisition unit is shutting downERROR [Force Acq Listener] c.o.s.a.f.m.ServiceManager (ServiceManager.java:501) - Acquisition is shutting down!ERROR [Force Acq Listener] c.o.s.a.f.m.ServiceManager (ServiceManager.java:470) - Data Source Events Manager reported an error: Failed to listen to acquisition events! com.onaro.commons.exception.ConfigurationException: Error Response 500 Internal Server Error null at com.onaro.commons.framework.mgmt.HTTPClientManager.parse(HTTPClientManager.java:48) ~[commons-commons.jar:9.18.0-2026.01.J68] at com.onaro.management.httpmanager.ServerMessagesHttpClientManager.listenForAUNotifierMessages(ServerMessagesHttpClientManager.java:50) ~[server-core.jar:9.18.0-2026.01.J68] at com.onaro.sanscreen.acquisition.framework.mgmt.DataSourceEventsManager$ForceAcqListenerThread.doRun(DataSourceEventsManager.java:69) ~[au-framework.jar:9.18.0-2026.01.J68] at com.onaro.sanscreen.acquisition.framework.mgmt.FrameworkThread.run(FrameworkThread.java:117) ~[au-framework.jar:9.18.0-2026.01.J68]Activ