ESXi host iSCSI datastore paths down due to duplicate server IP
Applies to
- ONTAP 9
- iSCSI
- VMware ESXi
Issue
- Path redundancy to storage device degraded logged in
vobd.log
of ESXi support bundle:
[esx.problem.storage.redundancy.degraded] Path redundancy to storage device naa.600a09803831434e6d2451365350636d degraded. Path vmhba64:C2:T0:L2 is down. Affected datastores: "VMFS_PROD".
[esx.problem.storage.redundancy.degraded] Path redundancy to storage device naa.600a09803831434e6d2451365350636d degraded. Path vmhba64:C4:T0:L2 is down. Affected datastores: "VMFS_PROD".
[esx.problem.storage.redundancy.degraded] Path redundancy to storage device naa.600a09803831434e6d2451365350636d degraded. Path vmhba64:C6:T0:L2 is down. Affected datastores: "VMFS_PROD".
- In ESXi Support Bundle,
esxcfg-mpath_-b.txt
shows some paths to a datastore are marked Dead:
naa.600a09803831434e6d2451365350636d : NETAPP iSCSI Disk (naa.600a09803831434e6d2451365350636d)
vmhba64:C1:T0:L2 LUN:2 state:active iscsi Adapter: iqn.1992-01.com.cisco:esxiserver Target: IQN=iqn.1992-08.com.netapp:sn.15ef1d64094811eb824100a0984f2e2c:vs.7 Alias= Session=00023d000018 PortalTag=1043
vmhba64:C2:T0:L2 LUN:2 state:dead iscsi Adapter: Unavailable Target: Unavailable
vmhba64:C11:T0:L2 LUN:2 state:active iscsi Adapter: iqn.1992-01.com.cisco:esxiserver Target: IQN=iqn.1992-08.com.netapp:sn.15ef1d64094811eb824100a0984f2e2c:vs.7 Alias= Session=00023d000012 PortalTag=1037
vmhba64:C4:T0:L2 LUN:2 state:dead iscsi Adapter: Unavailable Target: Unavailable
vmhba64:C6:T0:L2 LUN:2 state:dead iscsi Adapter: Unavailable Target: Unavailable
vmhba64:C9:T0:L2 LUN:2 state:active iscsi Adapter: iqn.1992-01.com.cisco:esxiserver Target: IQN=iqn.1992-08.com.netapp:sn.15ef1d64094811eb824100a0984f2e2c:vs.7 Alias= Session=00023d000014 PortalTag=1038
- Also seen in the
vobd.log
from the Support Bundle, messages indicating a duplicate IP address and specifying the vmk interface calling out the duplicate address:
[vob.net.vmknic.ip.duplicate] A duplicate IP address was detected for 172.1.21.11 on interface vmk3. The current owner is xx:xx:xx:xx:xx:xx.
[esx.problem.net.vmknic.ip.duplicate] Duplicate IP address detected for 172.1.21.11 on interface vmk3, current owner being xx:xx:xx:xx:xx:xx.
- It was noted that the affected channels, e.g.
C2
,C4
,C6
above, are all linked to the same vmkernel interface and IP address - Multiple LUNs are affected on the same host
- On other hosts, the LUNs have no path issues
- Upon a rescan or host reboot, the paths are active for a few minutes, then go dead again
- NetApp EMS shows new iSCSI sessions from the same IP address, but different IQNs:
iscsi.notice: ISCSI: New session from initiator iqn.1992-01.com.cisco:esxiserver at IP addr 10.0.0.1
iscsi.notice: ISCSI: New session from initiator iqn.1988-12.com.linux:linuxserver at IP addr 10.0.0.1