Getting data from AIQUM through REST API fails caused by service down
Applies to
ActiveIQ Unified Manager (AIQUM)
Issue
- AIQUM service stops.
- REST API does not return values correctly
acq.log
has several occcurence of "Data source is already performing a poll of netappstorageperformance. Therefore this poll is skipped.
"au.log
shows the following errors:
ERROR [main] c.o.s.a.f.Main (Main.java:210) - Main - Failed to signal acquisition service to stop... java.net.ConnectException: Connection refused (Connection refused)
at java.net.PlainSocketImpl.socketConnect(Native Method) ~[?:?]
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:412) ~[?:?]
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:255) ~[?:?]
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:237) ~[?:?]
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) ~[?:?]
at java.net.Socket.connect(Socket.java:609) ~[?:?]
at java.net.Socket.connect(Socket.java:558) ~[?:?]
at java.net.Socket.<init>(Socket.java:454) ~[?:?]
at java.net.Socket.<init>(Socket.java:264) ~[?:?]
at com.onaro.sanscreen.acquisition.framework.Main.shutDown(Main.java:199) ~[au-framework.jar:9.13.0-2023.09.J299]
at com.onaro.sanscreen.acquisition.framework.Main.main(Main.java:130) ~[au-framework.jar:9.13.0-2023.09.J299]
journalctl log
shows the following errors:
kernel: oom-kill:constraint=CONSTRAINT_NONE,nodemask=(null),cpuset=/,mems_allowed=0,global_oom,task_memcg=/system.slice/ocieau.service,task=java,pid=7651,uid=999
kernel: Out of memory: Killed process 7651 (java) total-vm:5469292kB, anon-rss:3027052kB, file-rss:0kB, shmem-rss:0kB, UID:999 pgtables:7016kB oom_score_adj:0
systemd[1]: ocieau.service: A process of this unit has been killed by the OOM killer.
ocieau[732614]: Platform installed at /opt/netapp/essentials
ocieau[732614]: stop ocieau
ocieau[732614]: Stopping Active IQ Acquisition Unit: ocieau.
ocieau[732614]: Waiting for ocieau to shut down.
sudo[732634]: root : PWD=/ ; USER=jboss ; COMMAND=/usr/bin/java -Dlog4j.configurationFile=/opt/netapp/essentials/au/conf/preferences/logconfig/ocie-production.xml -Donaro.home=/opt/netapp/essentials -Dcodebase= https://localhost:443 -Djava.library.path=/opt/netapp/essentials/au/bin -Dacq.home=/opt/netapp/essentials/au -Djava.security.properties=/opt/netapp/essentials/au/conf/java.security -Xmx2163m -Xms256m -XX:MaxMetaspaceSize=256m -classpath /opt/netapp/essentials/au/conf:/opt/netapp/essentials/au/*:/opt/netapp/essentials/au/lib/*:/opt/netapp/essentials/au/lib/xml/xerces/*:/opt/netapp/essentials/au/lib/xml/jdom/*:/opt/netapp/essentials/au/lib/jboss/client/*:/opt/netapp/essentials/au/lib/xml-3rd/*:/opt/netapp/essentials/au/lib/shared/*:/opt/netapp/essentials/au/../jboss/common/lib/* -Dno.trap=false -Djava.net.preferIPv4Stack=true -cp
sudo[732634]: root : (command continued) /opt/netapp/essentials/au/conf:/opt/netapp/essentials/au/*:/opt/netapp/essentials/au/lib/*:/opt/netapp/essentials/au/lib/xml/xerces/*:/opt/netapp/essentials/au/lib/xml/jdom/*:/opt/netapp/essentials/au/lib/jboss/client/*:/opt/netapp/essentials/au/lib/xml-3rd/*:/opt/netapp/essentials/au/lib/shared/*:/opt/netapp/essentials/au/../jboss/common/lib/* com.onaro.sanscreen.acquisition.framework.Main Stop
sudo[732634]: pam_unix(sudo:session): session opened for user jboss(uid=999) by (uid=0)
systemd[1]: ocieau.service: Main process exited, code=killed, status=9/KILL
sudo[732634]: pam_unix(sudo:session): session closed for user jboss
ocieau[732614]: .
systemd[1]: ocieau.service: Failed with result 'oom-kill'.
systemd[1]: ocieau.service: Consumed 3w 6d 9h 38min 59.403s CPU time.
systemd[1]: ocieau.service: Scheduled restart job, restart counter is at 1.
systemd[1]: Stopped LSB: Active IQ Acquisition Unit.
systemd[1]: ocieau.service: Consumed 3w 6d 9h 38min 59.403s CPU time.
systemd[1]: Starting LSB: Active IQ Acquisition Unit...
ocieau[732665]: Platform installed at /opt/netapp/essentials
ocieau[732665]: start ocieau
ocieau[732665]: Starting Active IQ Acquisition Unit: ocieau.
systemd[1]: Started LSB: Active IQ Acquisition Unit.