Brocade switch port flaps due to faulty end device sfp
Applies to
- All Brocade switch hardware platforms
- All Brocade Fabric Operating System (FOS) firmware levels
Issue
- Link resets reported under
Fabriclog
output as shown below:
Switch 0; Mon Dec 12 23:02:33 2022 IST (GMT+5:30)
23:02:33.251899 SCN Port Offline;rsn=0x3,g=0x4574 A2,P0 A2,P0 141 NA
23:02:33.251907 *Removing all nodes from port A2,P0 A2,P0 141 NA
23:02:34.716692 SCN LR_PORT(0);g=0x4574 A2,P0 A2,P0 141 NA
23:02:34.776188 SCN Port Online; g=0x4574,isolated=0 A2,P0 A2,P1 141 NA
23:02:34.777511 Port Elp engaged A2,P1 A2,P0 141 NA
23:02:34.777531 *Removing all nodes from port A2,P0 A2,P0 141 NA
23:02:34.777762 SCN Port F_PORT A2,P1 A2,P0 141 NA
23:02:37.875677 SCN Port Offline;rsn=0x3,g=0x4576 A2,P0 A2,P0 141 NA
23:02:37.875685 *Removing all nodes from port A2,P0 A2,P0 141 NA
23:02:39.656807 SCN LR_PORT(0);g=0x4576 A2,P0 A2,P0 141 NA
23:02:39.731303 SCN Port Online; g=0x4576,isolated=0 A2,P0 A2,P1 141 NA
23:02:39.733589 Port Elp engaged A2,P1 A2,P0 141 NA
23:02:39.733610 *Removing all nodes from port A2,P0 A2,P0 141 NA
23:02:39.733843 SCN Port F_PORT A2,P1 A2,P0 141 NA
- Switch port status shows as "online" under
switchshow
:
/fabos/bin/switchshow : switchName: XXXX switchType: 166.0 switchState: Online LS Attributes: [FID: 128, Base Switch: No, Default Switch: Yes, Ficon Switch: No, Address Mode 0] Index Slot Port Address Media Speed State Proto ============================================================ 141 3 29 668d40 id N8 Online FC F-Port 21:00:00:24:ff:yy:xx:xx
- Link reset seen under
Fabriclog
on switch-
15:38:51.649933 SCN LR_PORT(0);g=0x9fe LR_IN A2,P0 A2,P0 118 NA
16:16:45.053105 SCN LR_PORT(0);g=0x9fe LR_IN A2,P0 A2,P0 118 NA
16:34:18.013690 SCN LR_PORT(0);g=0x9fe LR_IN A2,P0 A2,P0 118 NA
Errdump
reportslink reset
andframe timeout
on the port-
2024/07/11-09:09:57 (IST), [C4-1014], 2442642, SLOT 1 | CHASSIS | PORT 12/6, WARNING, switch, Link Reset on Port S12,P118(63) vc_no=0 crd(s)lost=0 auto trigger.
2024/07/11-09:09:57 (IST), [C4-1014], 2442642, SLOT 1 | CHASSIS | PORT 12/6, WARNING, switch, Link Reset on Port S12,P118(63) vc_no=0 crd(s)lost=0 auto trigger.
2024/07/11-09:38:43 (IST), [AN-1014], 2443024, SLOT 1 | FID 128, INFO, switch, Frame timeout detected, tx port 12/6 rx port 12/29, sid 6b52xx, did 6b76xx, timestamp
Enc_out
errors are reported underporterrshow
:
/fabos/cliexec/porterrshow :
frames enc crc crc too too bad enc disc link loss loss frjt fbsy c3timeout pcs uncor
tx rx in err g_eof shrt long eof out c3 fail sync sig tx rx err err
141: 168 929 0 0 0 0 0 0 6.9k 0 0 0 0 0 0 0 0 0 0
crc err
andcrc g_eof
along withc3 tx discards
reported underporterrshow
-
frames enc crc crc too too bad enc disc link loss loss frjt fbsy c3timeout pcs uncor
tx rx in err g_eof shrt long eof out c3 fail sync sig tx rx err err
118: 5.2g 5.8g 0 5.2k 4.8k 14 0 408 0 2.2k 0 0 0 0 0 2.2k 0 559.2k 0
Portshow
reports high Invalid Transmitted Words (ITW) counter s reported
portshow 141
portDisableReason: None
portCFlags: 0x1
portFlags: 0x20b03 PRESENT ACTIVE F_PORT G_PORT U_PORT LOGICAL_ONLINE LOGIN NOELP ACCEPT FLOGI
LocalSwcFlags: 0x0
portType: 26.0
portState: 1 Online
Protocol: FC
portPhys: 6 In_Sync portScn: 32 F_Port
port generation number: 17782
state transition count: 24
portId: 668d40
portIfId: 43320035
portWwn: XX:8d:88:94:71:a2:XX.XX
portWwn of device(s) connected:
XX:00:00:24:ff:1d:xy:xx
Distance: normal
portSpeed: N8Gbps
Interrupts: 0 Link_failure: 0 Frjt: 0
Unknown: 0 Loss_of_sync: 0 Fbsy: 0
Lli: 0 Loss_of_sig: 0
Proc_rqrd: 0 Protocol_err: 0
Timed_out: 0 Invalid_word: 7186
Tx_unavail: 0 Invalid_crc: 0
Delim_err: 0 Address_err: 0
Lr_in: 0 Ols_in: 0
Lr_out: 0 Ols_out: 0
Cong_Prim_in: 0
Errdump
reportsRuleName=defALL_HOST_PORTSITW_21
getting trggered due to these ITW errors reporte din huge number.
Errdump:
2022/12/12-19:20:16, [MAPS-1003], 769558, SLOT 2 | FID 128, WARNING, switchXX, slot3 port29-Host.LINUX:CBXPR_APP02.ALIAS:HBCBXPRDAP02_HBA2:VSP5, F-Port 3/29, Condition=ALL_HOST_PORTS(ITW/min>21), Current Value:[ITW, 521 ITWs], RuleName=defALL_HOST_PORTSITW_21, Dashboard Category=Port Health.
2022/12/12-19:21:16, [MAPS-1003], 769561, SLOT 2 | FID 128, WARNING, switchXX, slot3 port29-Host.LINUX:CBXPR_APP02.ALIAS:HBCBXPRDAP02_HBA2:VSP5, F-Port 3/29, Condition=ALL_HOST_PORTS(ITW/min>21), Current Value:[ITW, 824 ITWs], RuleName=defALL_HOST_PORTSITW_21, Dashboard Category=Port Health.
2022/12/12-19:22:16, [MAPS-1003], 769562, SLOT 2 | FID 128, WARNING, switchXX, slot3 port29-Host.LINUX:CBXPR_APP02.ALIAS:HBCBXPRDAP02_HBA2:VSP5, F-Port 3/29, Condition=ALL_HOST_PORTS(ITW/min>21), Current Value:[ITW, 1422 ITWs], RuleName=defALL_HOST_PORTSITW_21, Dashboard Category=Port Health.
Sfpshow
reports low Rx power
sfpshow:
=============
Slot 3/Port 29:
=============
Identifier: 3 SFP
Connector: 7 LC
Transceiver: 6804406000000000 8,16,32_Gbps M5 sw Inter,Short_dist
Encoding: 6 64B66B
Baud Rate: 280 (units 100 megabaud)
Length 9u: 0 (units km)
Length 9u: 0 (units 100 meters)
Length 50u (OM2): 2 (units 10 meters)
Length 50u (OM3): 7 (units 10 meters)
Length 62.5u:0 (units 10 meters)
Length 50u (OM4): 10 (units 10 meters)
Vendor Name: BROCADE
Vendor OUI: 00:05:1e
Vendor PN: 57-1000333-01
Alarm flags[0,1] = 0x0, 0x0
Warn Flags[0,1] = 0x0, 0x0
Temperature: 52 Centigrade
Current: 5.832 mAmps
Voltage: 3322.6 mVolts
RX Power: -8.8 dBm (130.5uW)
TX Power: -1.3 dBm (743.3 uW)