Acquisition failed for all clusters in AIQUM due to maximum connection limit exhaustion
Applies to
- ActiveIQ Unified Manager (AIQUM) 9.6+
- All OS platforms
- ONTAP 9.x
Issue
- Intermittently acquisition is failing for all clusters added to AIQUM
Cluster Monitoring Failed
andCluster Not Reachable
alerts are triggered by AIQUM- However, the acquisition starts working automatically after sometime or if triggered manually
- All the prerequisites like AV exclusions and resources availability in terms of CPU/Memory/Disk space are applied on the AIQUM
- SSL certificates for AIQUM as well as the ONTAP clusters are valid
- AIQUM
au.log
:
ERROR [common-pool-2064] c.o.s.a.d.n.NetAppOCIEArchivePerformancePackage (NetAppOCIEArchivePerformancePackage.java:381) - Failed to get archive file names from zapi. java.net.SocketTimeoutException: connect timed out
at java.net.PlainSocketImpl.waitForConnect(Native Method) ~[?:?]
...
Wrapped by: com.onaro.sanscreen.acquisition.framework.datasource.DataSourceErrorException: Failed to connect to <cluster IP/Hostname>
at com.onaro.sanscreen.acquisition.datasource.netapp_ocie.transport.zapi.ZAPIConnection.createDefaultNaServer(ZAPIConnection.java:803) ~[au-datasource-netappfoundation.jar:9.13.0-2023.09.J299]
...
ERROR [common-pool-2064] c.o.s.a.f.d.BaseDataSource (DataSourceErrorException.java:246) - <cluster_IP/Hostname> [Error connecting] - Failed to connect to <cluster IP/Hostname> (connect timed out)
- AIQUM
ocumserver.log
:
ERROR [oncommand] [reconciliation-0] [c.n.d.c.ClusterStatusListener] Socket connection error for cluster: <cluster IP/Hostname> java.net.ConnectException: Connection timed out: connect
ERROR [oncommand] [reconciliation-0] [c.n.d.c.ClusterStatusListener] Cluster : <cluster IP/Hostname> is not reachable. Generating cluster not reachable event.