Acquisition server does not stay started on Oncommand Unified Manager 9.5 on RHEL 7.x or OVA
Applies to
- OnCommand Unified Manager 9.5
- RHEL 7.x
- Virtual Appliance
Issue
After an upgrade from Unified Manager 9.4 to 9.5 the "ocieau" service does not stay online.
OCUM / Oncommand Unified Manager 9.x is not able to monitor all the clusters, due to the acquisition services being down.
OCIEAU services does not stay online on RHEL 7.x
Acquisition stops after upgrade of Oncommand Unified Manager.
Check for the status of the ocieau services :
systemctl status ocieau.service
* ocieau.service - LSB: OnCommand Insight Essentials AU
Loaded: loaded (/opt/netapp/systemd/system/ocieau; enabled; vendor preset: disabled)
Active: inactive (dead) since Mon 2019-01-21 10:39:20 CET; 23min ago
Process: 16276 ExecStop=/opt/netapp/systemd/system/ocieau stop (code=exited, status=0/SUCCESS)
Process: 15656 ExecStart=/opt/netapp/systemd/system/ocieau start (code=exited, status=0/SUCCESS)
Main PID: 15656 (code=exited, status=0/SUCCESS)
Jan 21 10:37:53 %SystemName% systemd[1]: Starting LSB: OnCommand Insight Essentials AU...
Jan 21 10:37:53 %SystemName% ocieau[15656]: Starting NetApp OnCommand Insight Essentials Acqisition service ocieau
Jan 21 10:37:53 %SystemName% ocieau[15656]: NetApp OnCommand Insight Essentials Acquisition service started successfully
Jan 21 10:37:53 %SystemName% systemd[1]: Started LSB: OnCommand Insight Essentials AU.
Jan 21 10:39:20 %SystemName% systemd[1]: Stopping LSB: OnCommand Insight Essentials AU...
Jan 21 10:39:20 %SystemName% ocieau[16276]: NetApp OnCommand Insight Essentials Acquisition service already stopped
Jan 21 10:39:20 %SystemName% systemd[1]: Stopped LSB: OnCommand Insight Essentials AU.
In the OCUMserver.log there is a listing of
Waiting for next changelog update.
%datestamp% ERROR [oncommand] [pool-21-thread-3] [c.n.i.c.u.AcquisitionUnitStatusUtil] Unable to invoke AU EJB
java.io.IOException: Failed to retrieve RMIServer stub: javax.naming.ServiceUnavailableException [Root exception is java.rmi.ConnectException: Connection refused to host: localhost; nested exception is:
java.net.ConnectException: Connection refused (Connection refused)]
at javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:369)
at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:270)
at com.netapp.ipc.common.util.AcquisitionUnitStatusUtil.getAcquisitionUnitStartTime(AcquisitionUnitStatusUtil.java:42)
at com.netapp.ipc.common.util.AcquisitionUnitStatusUtil.isAcquisitionUnitRunning(AcquisitionUnitStatusUtil.java:30)
at com.netapp.scalemonitor.service.DiskMonitorService.monitor(DiskMonitorService.java:134)
at com.netapp.scalemonitor.ScaleMonitor.monitorDiskSpace(ScaleMonitor.java:35)
at com.netapp.scalemonitor.ScaleMonitorConfiguration$1.run(ScaleMonitorConfiguration.java:55)
at org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:54)
at org.springframework.scheduling.concurrent.ReschedulingRunnable.run(ReschedulingRunnable.java:81)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: javax.naming.ServiceUnavailableException: null
at com.sun.jndi.rmi.registry.RegistryContext.lookup(RegistryContext.java:136)
at com.sun.jndi.toolkit.url.GenericURLContext.lookup(GenericURLContext.java:205)
at javax.naming.InitialContext.lookup(InitialContext.java:417)
at javax.naming.InitialContext.lookup(InitialContext.java:417)
at javax.management.remote.rmi.RMIConnector.findRMIServerJNDI(RMIConnector.java:1955)
at javax.management.remote.rmi.RMIConnector.findRMIServer(RMIConnector.java:1922)
at javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:287)
... 15 common frames omitted
Caused by: java.rmi.ConnectException: Connection refused to host: localhost; nested exception is:
java.net.ConnectException: Connection refused (Connection refused)
at sun.rmi.transport.tcp.TCPEndpoint.newSocket(TCPEndpoint.java:619)
at sun.rmi.transport.tcp.TCPChannel.createConnection(TCPChannel.java:216)
at sun.rmi.transport.tcp.TCPChannel.newConnection(TCPChannel.java:202)
at sun.rmi.server.UnicastRef.newCall(UnicastRef.java:338)
at sun.rmi.registry.RegistryImpl_Stub.lookup(RegistryImpl_Stub.java:112)
at com.sun.jndi.rmi.registry.RegistryContext.lookup(RegistryContext.java:132)
... 21 common frames omitted
Caused by: java.net.ConnectException: Connection refused (Connection refused)
at java.net.PlainSocketImpl.socketConnect(Native Method)