Unable to add cluster to Unified Manager due to Reason: Error during websocket handshake: read failed
Applies to
AIQUM 9.14x+
Issue
- Unable to add cluster in unified manager.
- GUI Error:
Unable to add cluster datasource. This can occur if the clocks on
the systems are not synchronized and the Active IQ Unified Manager HTTPS certificate start date is later than the date on the cluster, or if the cluster has reached the maximum number of EMS notification destinations.
- Server_acq.log
[default task-6] c.n.u.RestUtil (RestUtil.java:411) - Job Failed: Failed to establish connection for cloud agent instance "UnifiedManager_c5376d62-9281-4951-8993-19b11b8da60a". Reason: AMQP transport failed for connection "UnifiedManager_c5376d62-9281-4951-8993-19b11b8da60a". Reason: Error during websocket handshake: read failed.
[JmsSession [ID:3433fcae-9a63-4444-ad2f-c3681e30c627:1:1] delivery dispatcher] c.n.s.m.JmsUtils (JmsUtils.java:189) - ONTAP cluster "Clustername" sent counters message, JMSXGroupID=24, JMSXDeliveryCount=1, cluster_uuid=16c6b7a7-0d59-11eb-afdb-d039ea1fed20, group_final=0, collection_mode=on_demand, name=UnifiedManager_c5376d62-9281-4951-8993-19b11b8da60a, JMSXGroupSeq=1, manifest_revision=1, uuid=4e56e3f4-a340-11ef-ba93-d039ea1fec38
[JmsSession [ID:3433fcae-9a63-4444-ad2f-c3681e30c627:1:1] delivery dispatcher] c.n.s.m.JmsUtils (JmsUtils.java:189) - ONTAP cluster "Clustername" sent counters message, JMSXGroupID=24, JMSXDeliveryCount=1, cluster_uuid=16c6b7a7-0d59-11eb-afdb-d039ea1fed20, group_final=0, collection_mode=on_demand, name=UnifiedManager_c5376d62-9281-4951-8993-19b11b8da60a, JMSXGroupSeq=2, manifest_revision=1, uuid=4e56e3f4-a340-11ef-ba93-d039ea1fec38
[JmsSession [ID:3433fcae-9a63-4444-ad2f-c3681e30c627:1:1] delivery dispatcher] c.n.s.m.JmsUtils (JmsUtils.java:189) - ONTAP cluster "Clustername" sent counters message, JMSXGroupID=24, JMSXDeliveryCount=1, cluster_uuid=16c6b7a7-0d59-11eb-afdb-d039ea1fed20, group_final=1, collection_mode=on_demand, name=UnifiedManager_c5376d62-9281-4951-8993-19b11b8da60a, JMSXGroupSeq=3, manifest_revision=1, uuid=4e56e3f4-a340-11ef-ba93-d039ea1fec38
[JmsSession [ID:3433fcae-9a63-4444-ad2f-c3681e30c627:1:1] delivery dispatcher] c.n.s.m.CollectionMessenger (CollectionMessenger.java:88) - Sent message to AU Listener for cluster: 16c6b7a7-0d59-11eb-afdb-d039ea1fed20 and filename: /var/log/ocie/recording/cloudagent/performance/"Clustername"/2024-11-15T120300+0100
[JmsSession [ID:3c5475f6-7eea-4768-a137-774d944d7a9f:1:1] delivery dispatcher] c.n.s.m.JmsUtils (JmsUtils.java:189) - ONTAP cluster DataCollectorTrigger sent counters.request message, start_time=10/31/2024 12:55:00, JMSXDeliveryCount=1, cluster_uuid=16c6b7a7-0d59-11eb-afdb-d039ea1fed20, datastore=opm, name=AIQ_UM_AU_TRIGGER, end_time=10/31/2024 13:10:00, uuid=c5376d62-9281-4951-8993-19b11b8da60a
[default task-6] c.n.u.CloudAgentConnectionUtil (CloudAgentConnectionUtil.java:223) - Failed to establish connection for cloud agent instance "UnifiedManager_c5376d62-9281-4951-8993-19b11b8da60a". Reason: AMQP transport failed for connection "UnifiedManager_c5376d62-9281-4951-8993-19b11b8da60a". Reason: Error during websocket handshake: read failed.
[default task-6] c.n.u.CloudAgentConnectionUtil (CloudAgentConnectionUtil.java:254) - Establishing connection with cluster "Clustername" failed
[default task-6] c.n.s.m.AccessHandle (AccessHandle.java:101) - Transitioning state from WAIT to DOWN
[default task-6] c.n.s.m.OntapMessenger (OntapMessenger.java:110) - Sent connection message to ONTAP for cluster: affe81e6-0ece-11eb-bf3f-d039ea1fec58
[default task-6] c.n.s.m.DataCollectorManager (DataCollectorManager.java:309) - Deleting cloud agent connection to cluster affe81e6-0ece-11eb-bf3f-d039ea1fec58
[default task-6] c.n.s.a.s.r.s.i.AcquisitionFacadeSessionServiceImpl (AcquisitionFacadeSessionServiceImpl.java:923) - Failed to establish connection for datasource id: 17, java.lang.RuntimeException: Failed to establish connection for cloud agent instance "UnifiedManager_c5376d62-9281-4951-8993-19b11b8da60a". Reason: AMQP transport failed for connection "UnifiedManager_c5376d62-9281-4951-8993-19b11b8da60a". Reason: Error during websocket handshake: read failed.
[default task-6] c.n.s.a.s.r.s.i.AcquisitionFacadeSessionServiceImpl (AcquisitionFacadeSessionServiceImpl.java:842) - Fail to add data source for acquisitionUnitName: local, dataSourceName: cjibhacl02, vendor:NetApp OCI Essentials, model: NetApp OCI Essentials, datasourceTypeId: 91, manual: 1, isActive: false, attrs: -- listing properties --
....
: java.lang.RuntimeException: java.lang.RuntimeException: java.lang.RuntimeException: Failed to establish connection for cloud agent instance "UnifiedManager_c5376d62-9281-4951-8993-19b11b8da60a". Reason: AMQP transport failed for connection "UnifiedManager_c5376d62-9281-4951-8993-19b11b8da60a". Reason: Error during websocket handshake: read failed.
at deployment.mega-app.ear.server-core.jar//com.netapp.sanscreen.acquisition.sessions.rest.service.impl.AcquisitionFacadeSessionServiceImpl.createCloudAgentConnection(AcquisitionFacadeSessionServiceImpl.java:928)
at deployment.mega-app.ear.server-core.jar//com.netapp.sanscreen.acquisition.sessions.rest.service.impl.AcquisitionFacadeSessionServiceImpl.addDS(AcquisitionFacadeSessionServiceImpl.java:817)
at deployment.mega-app.ear.server-api.war//com.netapp.sanscreen.acquisition.sessions.rest.service.impl.AcquisitionFacadeSessionServiceImpl$$FastClassBySpringCGLIB$$92eba61a.invoke(<generated>)
at deployment.mega-app.ear//org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)
at deployment.mega-app.ear//org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:793)
Caused by: java.lang.RuntimeException: java.lang.RuntimeException: Failed to establish connection for cloud agent instance "UnifiedManager_c5376d62-9281-4951-8993-19b11b8da60a". Reason: AMQP transport failed for connection "UnifiedManager_c5376d62-9281-4951-8993-19b11b8da60a". Reason: Error during websocket handshake: read failed.
at deployment.mega-app.ear.acqServer.jar//com.netapp.util.CloudAgentConnectionUtil.createConnection(CloudAgentConnectionUtil.java:255)
at deployment.mega-app.ear.acqServer.jar//com.netapp.util.CloudAgentConnectionUtil.initiateConnection(CloudAgentConnectionUtil.java:162)
at deployment.mega-app.ear.server-core.jar//com.netapp.sanscreen.acquisition.sessions.rest.service.impl.AcquisitionFacadeSessionServiceImpl.createCloudAgentConnection(AcquisitionFacadeSessionServiceImpl.java:905)
... 122 more
Caused by: java.lang.RuntimeException: Failed to establish connection for cloud agent instance "UnifiedManager_c5376d62-9281-4951-8993-19b11b8da60a". Reason: AMQP transport failed for connection "UnifiedManager_c5376d62-9281-4951-8993-19b11b8da60a". Reason: Error during websocket handshake: read failed.
at deployment.mega-app.ear.acqServer.jar//com.netapp.util.CloudAgentConnectionUtil.createConnection(CloudAgentConnectionUtil.java:224)
... 124 more