C3 discards and FRAME LOSS on Brocade switch due to faulty SFP on switch end
Applies to
- Brocade Fabric OS
- ONTAP
- Hyper-v Windows host
Issue
- ONTAP Target port connected to Brocade switch port errors with discards C3 ( disc c3 ) and C3 timeout TX
- Corresponding device or ONTAP SFP has been replaced with known good
- Cable has been swapped/replaced with known good
- Slow drain device reported
- Host experience disk IO errors and paused/hung IO
Example:
The system failed to flush data to the transaction log. Corruption may occur in VolumeId: xxxxxxxx, DeviceName: \Device\HarddiskVolume123.
({Data Overrun}
A data overrun error occurred - Ntfs
The IO operation at logical block address 0x1debb18 for Disk 0 (PDO name: \Device\0000002f) was retried
- disc c3 and c3 timeout tx discards
porterrshow
frames enc crc crc too too bad enc disc link loss loss frjt fbsy c3timeout pcs uncor
tx rx in err g_eof shrt long eof out c3 fail sync sig tx rx err err
352: 40.0m 183.4m 0 0 0 0 0 0 29.5k 6.3k 10 5 1 0 0 605 0 0 0
portstatsshow
portstatsshow 103
er_tx_c3_timeout 360 Class 3 transmit frames discarded due to timeout
- RX and TX in optimal range
sfpshow
Port 23:
=============
[...]
RX Power: -1.8 dBm (655.2uW)
TX Power: -2.1 dBm (618.8 uW)
-
IO Frame loss events observed along with C3 Tx timeout events in
errdump
CRITICAL, CDVL_X, slot11 port32, F-Port 11/32, Condition=ALL_OTHER_F_PORTS(C3TXTO/min>3), Current Value:[C3TXTO, 20 Timeouts], RuleName=defALL_OTHER_F_PORTSC3TXTO_3, Dashboard Category=Port Health.
CRITICAL, CDVL_X, slot11 port32, F-Port 11/32, Condition=ALL_PORTS(DEV_LATENCY_IMPACT==IO_FRAME_LOSS), Current Value:[DEV_LATENCY_IMPACT, IO_FRAME_LOSS, (20 C3TX Timeouts) ], RuleName=defALL_PORTS_IO_FRAME_LOSS_UNQUAR, Dashboard Category=Fabric Performance Impact.
WARNING, , Flow (SID=0x6cc0c0,DID=0x6c9bc1,Host Port=9/40), Condition=sys_flow_monitor(RD_STATUS_TIME/10SEC>12000), Current Value:[RD_STATUS_TIME, 223776 Microseconds], RuleName=defRD_STATUS_TIME_12000, Dashboard Category=IO Latency.
INFO, FAB1, Frame timeout detected, tx port 1/23 rx port 1/23, sid 123, did 456, timestamp xy
INFO, FAB1, Local domain 1, port index 123: A zoned device 0x123456 has been quarantined.
WARNING, FAB1, Port 1/23 (Port index 123) has been marked as Slow Drain Device.