ANDU paused due to hung BIOS upgrade
bug 1547361
Applies to
- FAS
- AFF
- Automated non-disruptive upgrade (ANDU).
Issue
- Post ONTAP upgrade when controller boots, the bios gets upgraded and stops as below
System log:
Initializing System Memory ...
Loading Device Drivers ...
BIOS image updated, resetting to load defaults.
BIOS Version: 11.9
Events all:
Record 1048: Mon Nov 09 18:45:21.610000 2020 [BMC.notice]: PCIE Config Info received from BIOS
Record 1049: Mon Nov 09 18:45:21.620000 2020 [BMC.notice]: ScratchPad Config Info received from BIOS
Record 1050: Mon Nov 09 18:45:21.710000 2020 [IPMI.notice]: 0088 | 02 | EVT: 6fc203ff | System_FW_Status | Assertion Event, "Memory Initialization done"
Record 1051: Mon Nov 09 18:45:29.000000 2020 [SysFW.notice]: SP responded to sync command in 0 seconds.
Record 1052: Mon Nov 09 18:45:29.000000 2020 [SysFW.notice]: BIOS Version: 11.9
Record 1053: Mon Nov 09 18:45:27.970000 2020 [IPMI.notice]: (PUA) Enable power to all PCIe slots
Record 1054: Mon Nov 09 18:45:27.970000 2020 [IPMI.notice]: (PUA) Enable power to all PCIe on board device
Record 1055: Mon Nov 09 18:45:28.030000 2020 [IPMI.notice]: (PUA) P_stat :slots=0x0,onboard_devs=0x0,final
Record 1056: Mon Nov 09 18:45:28.030000 2020 [IPMI.notice]: (PUA) Power status of all PCIe slots unchanged
Record 1057: Mon Nov 09 18:45:37.790000 2020 [IPMI.notice]: 0089 | 02 | EVT: 6fc220ff | System_FW_Status | Assertion Event, "Bootloader is running"
Record 1058: Mon Nov 09 18:45:41.810000 2020 [IPMI.notice]: 008a | 02 | EVT: 6fc22fff | System_FW_Status | Assertion Event, "Storage OS Running"
Record 1059: Mon Nov 09 18:53:41.770000 2020 [BMC.critical]: Heartbeat stopped
- EMS shows Ontap Upgrade is paused as below:
[node01-01: notifyd: callhome.andu.pausederr:alert]: params: {'epoch': '#######-####-####-####-############', 'subject': 'AUTOMATED NDU PAUSED'}
- The node may also exhibit the symptom wher the SP/BMC is unresponsive and not reachable via its configured IP address.