Brocade FC port offline with No_Sync state due to faulty end device sfp

Applies to

• Brocade FC Switch
• Fabric OS (FOS)

Issue

• Brocade fiber channel port is in No_sync , Disabled state under switchshow output: 

/fabos/bin/switchshow :
Index Slot Port Address Media  Speed        State    Proto
============================================================
120 4 24 6f7800 id N32 No_Sync FC Disabled (Port Throttled)

 152    4   24   0f9840   id    N32      No_Sync     FC  Disabled (Port State Change threshold exceeded)
 

 

• PCS error , uncorr error along with link fail and loss signal errors reported under porterrshow

porterrshow :
frames enc crc crc too too bad enc disc link loss loss frjt fbsy c3timeout pcs uncor
tx rx in err g_eof shrt long eof out c3 fail sync sig tx rx err err
152: 22.6k 15.2k 0 0 0 0 0 0 0 0 16 17 17 0 0 0 0 4.1k 1.1k
 

• Rule defALL_HOST_PORTSITW_21 reported on the affected switch port under errdump:

2023/01/06-01:18:52, [MAPS-1003], 207782, SLOT 2 | FID 128, WARNING, switch, slot12 port37, F-Port 12/37, Condition=ALL_HOST_PORTS(ITW/min>21), Current Value:[ITW, 5616 ITWs], RuleName=defALL_HOST_PORTSITW_21, Dashboard Category=Port Health.
2023/01/06-16:45:46, [MAPS-1003], 208332, SLOT 2 | FID 128, WARNING, switch, slot12 port37, F-Port 12/37, Condition=ALL_HOST_PORTS(ITW/min>21), Current Value:[ITW, 1764 ITWs], RuleName=defALL_HOST_PORTSITW_21, Dashboard Category=Port Health.
 

• sfpshow reports Tx and Rx power in recommended range

=============
Slot  4/Port  24:
=============

RX Power:    -0.6    dBm (861.3uW)
TX Power:    -0.3    dBm (933.0 uW)

• Due to the port in Disabled state, the sfpshow output reports very low Tx power with Rx power near the threshold value.

Slot  4/Port 24:
=============
RX Power:    -3.5    dBm (451.1uW)
TX Power:    -40.0   dBm (0.1   uW)

• sfpshow reports low Rx and Tx power as -inf:

=============
Slot 12/Port 37:
=============
RX Power: -8.8 dBm (132.0uW)
TX Power: -inf dBm (0.0 uW)

 

• Porterrshow does not report any errors on the switch port.

/fabos/cliexec/porterrshow:
          frames        enc     crc     crc     too     too     bad     enc    disc    link    loss    loss    frjt    fbsy     c3timeout     pcs      uncor
        tx       rx      in     err     g_eof   shrt    long    eof     out    c3      fail    sync    sig                      tx      rx      err     err
238:    0       0       0       0       0       0       0       0       0       0       0       0       0       0       0       0       0       0       0

 

• sfpshow reports Tx power in the recommended range, however, the Rx power reported is low.

RX Power: -9.9 dBm (102.6uW)
TX Power: -0.8 dBm (837.1 uW)

 

• Multiple link resets reported under errdump:

2022/11/11-18:22:58, [C5-1014], 17914, SLOT 1 | CHASSIS, WARNING, XXX,  Link Reset on Port S6,P296(120) vc_no=3 crd(s)lost=1 auto trigger.
2022/11/11-18:29:41, [C5-1040], 17915, SLOT 1 | CHASSIS, WARNING, XXX, Multi RDY/Frame Loss detected on Slot 6, Port 296(120) m_rdy(0x0)/m_frame(0x1). Link Reset done.
2022/11/11-18:29:41, [C5-1014], 17916, SLOT 1 | CHASSIS, WARNING, XXX,  Link Reset on Port S6,P296(120) vc_no=0 crd(s)lost=1 auto trigger.
2022/11/11-18:39:24, [C5-1023], 17918, SLOT 1 | CHASSIS, WARNING, XXX, Single RDY/Frame Loss detected and recovered on Slot 6,Port 296(120) rdy(0x0)/frame(0x1).
2022/11/11-18:43:46, [C5-1040], 17919, SLOT 1 | CHASSIS, WARNING, XXX, Multi RDY/Frame Loss detected on Slot 6, Port 296(120) m_rdy(0x0)/m_frame(0x1). Link Reset done.
2022/11/11-18:43:46, [C5-1014], 17920, SLOT 1 | CHASSIS, WARNING, XXX,  Link Reset on Port S6,P296(120) vc_no=0 crd(s)lost=1 auto trigger.

 

• Connected NetApp port to the Brocade switch shows the status as "Link Disconnected"

 slot 0: Fibre Channel Target Host Adapter 0e
                (QLogic 8324 (8362) rev. 2, <LINK DISCONNECTED>)
                Firmware rev:      8.9.0
                Flash rev:         5.0.2
                Host Port Addr:    000000

• NetApp port connected to the switch port  shows Rx and Tx: 0.0 (uWatts)

 

• Brocade switch port reports "No_Light" status.

Index Slot Port Address Media  Speed        State    Proto
============================================================
 202    9   26   01ca00   id    N32  No_Light    FC

• portshow output on switch reports portstate as "Offline".

portshow 202
portDisableReason: None
portCFlags: 0x1
portFlags: 0x41PRESENT
LocalSwcFlags: 0x0
portType:  26.0
portState: Offline
Protocol: FC
portPhys:  No_Light

 

• Below are some of the troubleshooting steps that should be followed:

  • `Verify if the switch end sfp Tx and Rx power are within the recommended range.

  • For fault isolation, swap the switch end sfp and perform cable test.

  • If issues persists, then end device has to be reviewed.Engage host vendor for assistance once the switch end sfp has been replaced and cable has been tested.

  • Check the HBA/host and confirm what speed the HBA is capable of running and try to set the speed of the switch port accordingly (manually) using portCfgSpeed command and check.