Brocade Fabric OS update fails due to timeout

Applies to

Brocade FabricOS (FOS) 

Issue

• SANnav FOS upgrade showed a time out message:

SWITCH:admin> firmwaredownloadstatus

[1]: Sat Sep 14 23:36:11 2024
Slot 1 (CP0, active): Firmware is being downloaded to standby CP. This step may take up to 30 minutes.

[2]: Sun Sep 15 00:06:11 2024
Slot 1 (CP0, active): Firmware download failed on standby CP - Firmware download timed out. (0x23)

[3]: Sun Sep 15 00:06:20 2024
Slot 1 (CP0, active): Please use firmwaredownloadstatus and firmwareshow to see the firmware status.

• Switch appears to be busy with several MAPs alerts for flow impact: 

2023/04/10-22:10:36:223765 (IST), [MAPS-1005], 428700/228252, SLOT 2 | FID 128, WARNING, Brocade, Flow (SID=0x6a2xx0,DID=0x66ayy0,Host Port=5/14), Condition=sys_flow_monitor_scsi(RD_1stDATA_TIME/10SEC>11000), Current Value:[RD_1stDATA_TIME, 11239 Microseconds], Rule defRD_1stDATA_TIME_11000 triggered 2 times in 10 min and last trigger time Mon Apr 10 22:07:37 2023, Dashboard Category=IO Latency., raslogAction.c, line: 233, comp:md, ltime:2023/04/10-22:10:36:223568

2023/04/10-22:10:36:224539 (IST), [MAPS-1006], 428701/228253, SLOT 2 | FID 128, INFO, Brocade, Flow (SID=0x6a2xx0,DID=0x66ayy0,Host Port=5/14), Latest Value=[RD_1stDATA_TIME, 11239 Microseconds]., raslogAction.c, line: 244, comp:md, ltime:2023/04/10-22:10:36:223703

2023/04/10-22:11:36:225540 (IST), [MAPS-1005], 428702/228254, SLOT 2 | FID 128, WARNING, Brocade, Flow (SID=0x6a2xx0,DID=0x66ayy0,Host Port=12/10), Condition=sys_flow_monitor_scsi(RD_1stDATA_TIME/10SEC>11000), Current Value:[RD_1stDATA_TIME, 15935 Microseconds], Rule defRD_1stDATA_TIME_11000 triggered 2 times in 10 min and last trigger time Mon Apr 10 22:01:17 2023, Dashboard Category=IO Latency., raslogAction.c, line: 233, comp:md, ltime:2023/04/10-22:11:36:225343

2023/04/10-22:11:36:226424 (IST), [MAPS-1006], 428703/228255, SLOT 2 | FID 128, INFO, Brocade, Flow (SID=0x6a2xx0,DID=0x66ayy0,Host Port=12/10), Latest Value=[RD_1stDATA_TIME, 15935 Microseconds]., raslogAction.c, line: 244, comp:md, ltime:2023/04/10-22:11:36:225480

2023/04/10-22:11:36:226465 (IST), [MAPS-1005], 428704/228256, SLOT 2 | FID 128, WARNING, Brocade, Flow (SID=0x6a2xx0,DID=0x66ayy0,Host Port=12/10), Condition=sys_flow_monitor_scsi(RD_STATUS_TIME/10SEC>12000), Current Value:[RD_STATUS_TIME, 15941 Microseconds], Rule defRD_STATUS_TIME_12000 triggered 2 times in 10 min and last trigger time Mon Apr 10 22:01:17 2023, Dashboard Category=IO Latency., raslogAction.c, line: 233, comp:md, ltime:2023/04/10-22:11:36:226109

2023/04/10-22:11:36:226499 (IST), [MAPS-1006], 428705/228257, SLOT 2 | FID 128, INFO, Brocade, Flow (SID=0x6a2xx0,DID=0x66ayy0,Host Port=12/10), Latest Value=[RD_STATUS_TIME, 15941 Microseconds]., raslogAction.c, line: 244, comp:md, ltime:2023/04/10-22:11:36:226243

• Error code 0x23 translates to the error message: Firmware download timed out as per BROCADE FABRIC OS MESSAGE REFERENCE MANUAL, 9.1.X (page no:1231)

Refer: Fabric OS System Messages

• These MAP events can cause issues downloading the code and copying it internally.
• Check if there are any network issues observed during the upgrade.

 

• Below steps can be followed to increase the timeout value-
  • Wait until the firmwareDownload operation completes (about 30 minutes).
  • Increase the timeout to 40 min with  firmwareDownload -x 2400
  • In case it does not complete, run the firmwareDownload -s command to upgrade the firmware on the standby CP to the same version as the active CP. Then retry the firmwareDownload command to download the desired version of firmware onto the CPs.

 

• In case, the above steps does not resolve the issue, please proceed with the alternative steps mentioned in the solution.