ONTAP Select: SYNCMIRROR PLEX FAILED due to loss of connectivity of backing storage
Applies to
- ONTAP Select (OTS)
- VMware ESXi
Issue
- The OTS cluster triggers a SYNCMIRROR PLEX FAILED alert due to a node within the HA pair rebooting unexpectedly.
- From the host logs, more specifically the vmware.log, we can see heartbeat failures around the time of the SYNCMIRROR PLEX failure alert from ASUPS.
2025-08-01T14:23:48.306Z In(05) vcpu-0-Tools: Tools heartbeat timeout.
2025-08-01T14:23:48.306Z In(05) vmx-Tools: [AppStatus] Last heartbeat value 9020915 (last received 29s ago)
2025-08-01T14:23:17.243Z: [vmfsCorrelator] 9021840650053us: [vob.vmfs.heartbeat.timedout] 63334286-6a32c6e0-0285-88e9a45210ce iscsi1
2025-08-01T14:23:17.244Z: [vmfsCorrelator] 9022040343959us: [esx.problem.vmfs.heartbeat.timedout] 63334286-6a32c6e0-0285-88e9a45210ce iscsi1
2025-08-01T14:23:48.301Z: [vmfsCorrelator] 9021871707338us: [vob.vmfs.heartbeat.recovered] Reclaimed heartbeat for volume 63334286-6a32c6e0-0285-88e9a45210ce (iscsi1): [Timeout] [HB state abcdef02 offset 3932160 gen 851 stamp US 9021871707267 uuid 68032397-b0a51d88-376b-88e9a45210ce jrnl drv 24.82]
2025-08-01T14:23:48.301Z: [vmfsCorrelator] 9022071401635us: [esx.problem.vmfs.heartbeat.recovered] 63334286-6a32c6e0-0285-88e9a45210ce iscsi1
- From hostd.log within the ESXi host log bundle, we can see that the backing volume (iscsi1) mounted to the host which the affected OTS node resides on, losses connectivity to the host at the time of the SYNCMIRROR PLEX FAILED alert.
2025-08-01T14:23:17.245Z info hostd[2102547] [Originator@6876 sub=Vimsvc.ha-eventmgr] Event 601013 : Lost access to volume 63334286-6a32c6e0-0285-88e9a45210ce (iscsi1) due to connectivity issues. Recovery attempt is in progress and outcome will be reported shortly.
2025-08-01T14:23:36.416Z info hostd[2102496] [Originator@6876 sub=Vmsvc.ProductLockerWatcher] Failed to list directory /usr/lib/vmware/isoimages, No such file or directory
