CFBMC-3569: BMC "heartbeat stopped" event causes node shutdown
Issue
- On AFF A900, ASA A900 or FAS9500 systems, ONTAP might detect the loss of BMC heartbeat and initiate a system shutdown or emergency recovery to prevent hardware damage or data loss.
- Example events indicating the issue:
[node-01: spmgrd: sp.heartbeat.stopped:info]: Have not received a IPMI heartbeat from the Service Processor (SP) in last 600 seconds.
[node-01: spmgrd: callhome.sp.hbt.missed:notice]: Call home for SP HBT MISSED
[node-01: spmgrd: callhome.sp.hbt.stopped:alert]: Call home for SP HBT STOPPED
[node-01: env_mgr: sp.ipmi.lost.shutdown:EMERGENCY]: SP heartbeat stopped and cannot be recovered. To prevent hardware damage and data loss, the system will shut down in 10 minutes.
[node-01: env_mgr: monitor.shutdown.emergency:EMERGENCY]: Emergency shutdown: Environmental Reason Shutdown (System reboot to recover the SP)- When the BMC becomes unresponsive, ONTAP reboots to attempt a power-cycle of the BMC to recover it.
