Path redundancy degraded alerts on ESXi host due to faulty host sfp on switch
Applies to
- Ontap 9
- FC
- ESXi
- Brocade SAN Switch
Issue
-
Multiple NetApp LUNs were seeing host connectivity issues along with degraded paths and frequent disconnections of NetApp FC luns.
- ESXi host reports below alert on VMware side on the vCenter-
This email is to notify you that an alarm has been triggered in your vCenter:[Warning] Alarm alarm.StorageConnectivityAlarm on Host hostabc.xxx.combecause Path redundancy to storage device naa.600a098xxxxxx46c3f515xxxxxxxx degraded. Path vmhba2:C0:xx:xx0 is down. Affected datastores: xxx-NetApp-xyz..Alarm name alarm.StorageConnectivityAlarmDescription alarm.StorageConnectivityAlarmTarget Host hostabc.xxx.comStatus Warning (previous status: Normal)Triggered time 04/03/2024 01:27:05 PMPath redundancy to storage device naa.600a098xxxxxx46c3f515xxxxxxxx degraded. Path vmhba2:C0:T8:L142 is down. Affected datastores: xxx-NetApp-xyz. Warning 04/04/2024, 11:12:40 AM -
LUNs on storage side are online and mapped.
-
FC ports are all up and Rx, Tx values are in optimal range.
-
STIO hung cmd events with state=5 reported in EMS :
Wed Apr 03 13:02:34 +0200 [NetApp: fct_tpd_thread_5: fcp.io.status:debug]: STIO Adapter:0g, found hung cmd:0xfffff808ed70a770(state=5, flags=0x0, ctio_sent=1/1,RecvExAddr=0x1217d0, OX_ID=0x125, RX_ID=0xffff,SID=0x4105xx, Cmd[2A], req_q_free:3501)
Wed Apr 03 14:41:09 +0200 [NetApp: fct_tpd_thread_4: fcp.io.status:debug]: STIO Adapter:0h, found hung cmd:0xfffff808ed1d8b38(state=5, flags=0x0, ctio_sent=1/1,RecvExAddr=0x11d570, OX_ID=0x735, RX_ID=0xffff,SID=0x4105xx, Cmd[2A], req_q_free:1321)
NOTE: “state=5: DATAOUT_WAIT - This indicates that the FC target is waiting for something to come back from the host after accepting the write request; however, nothing came back within the expected timeout value.”
- These STIO events were coming from two specific SIDs.
- The host connected port on the SAN switch has the status of Laser_FLT, which indicates faulty sfp.
Index Slot Port Address Media Speed State Proto
============================================================
5 1 5 701400 id N16 Laser_Flt FC