A400/FAS8300/FAS8700 BMC offline during FW upgrade
Applies to
- AFF-A400
- FAS8300
- FAS8700
- BMC FW(firmware) version 13.6P1 or earlier
Issue
- During a BMC FW upgrade, which requires a reboot of the BMC, it can hang, which could lead to an ONTAP shutdown event.
- When the BMC hangs, both the BMC network and the node's remote console access are lost, and require physical console access to reboot the BMC.
Example:
7/15/2022 14:38:25 nodename NFORMATIONAL sp.heartbeat.resumed: Received IPMI heartbeat from the Service Processor (SP).
7/15/2022 14:37:45 nodename NOTICE callhome.sp.hbt.missed: Call home for SP HBT MISSED
7/15/2022 14:37:45 nodename ERROR sp.heartbeat.stopped: Have not received a IPMI heartbeat from the Service Processor (SP) in last 600 seconds.
7/15/2022 14:32:00 nodename ALERT hm.alert.raised: Alert Id = SPAutoUpgradeFailedMajorAlert , Alerting Resource = SP Upgrade raised by monitor controller
7/15/2022 14:27:54 nodename ERROR sp.servprocd.upd.error: SP update error: SP firmware update failure has been detected.
7/15/2022 14:27:54 nodename DEBUG sp.servprocd.upd.unexpt.evts: reason="BMC update - Update has timed out. Last status code (8)"
7/15/2022 14:26:01 nodename ERROR sp.heartbeat.stopped: Have not received a IPMI heartbeat from the Service Processor (SP) in last 600 seconds.
7/15/2022 14:12:54 nodename DEBUG sp.servprocd.upd.evts: reason="SP Firmware network update from 13.5 to 13.6P1 has been triggered."