Automatic SP upgrade fails after ONTAP upgrade on AFF A300 and FAS8200
Applies to
- AFF-A300
- FAS8200
- ONTAP 9
- SP versions prior to 5.9
Issue
- After completing ONTAP upgrade bundled with newer SP firmware, the automatic SP upgrade fails and remains on previous version.
::> event log show -node * -message-name sp.servprocd.upd.error
Node3 ERROR sp.servprocd.upd.error: SP update error: SP firmware update failure has been detected. Node2 ERROR sp.servprocd.upd.error: SP update error: SP firmware update failure has been detected. Node6 ERROR sp.servprocd.upd.error: SP update error: SP firmware update failure has been detected. Node5 ERROR sp.servprocd.upd.error: SP update error: SP firmware update failure has been detected.
SP-MGMT-MLOG-TXT
log (Can be viewed from Active IQ digital Advisor)
[kern_servprocd:info:6905] 0x809878700: 0: NOTICE: Servprocd::SpUpdate: update: SP firmware image for network transfer has been created.
kern_servprocd:info:6905] 0x809878700: 0: NOTICE: Servprocd::SpUpdate: transferImage: sending /mroot/etc/RLM_FW/sp_image.tar.gz to SP, destination: /tmp/sp_image.tar.gz.
[kern_servprocd:info:6905] 0x809878700: 0: ERR: Servprocd::SpUpdate: transferImage: FTM sp_send_file failed to transfer /mroot/etc/RLM_FW/sp_image.tar.gz to SP. Error 26. Sleep for 30 secs and retry. Retry count : 1
[kern_servprocd:info:6905] 0x809878700: 0: NOTICE: Servprocd::SpUpdate: transferImage: sending /mroot/etc/RLM_FW/sp_image.tar.gz to SP, destination: /tmp/sp_image.tar.gz.
[kern_servprocd:info:6905] 0x809878700: 0: ERR: Servprocd::SpUpdate: transferImage: FTM sp_send_file failed to transfer /mroot/etc/RLM_FW/sp_image.tar.gz to SP. Error 26. Sleep for 30 secs and retry. Retry count : 2
- EMS logs show below alerts:
nphmd hm alert raised: detailed_info="Alert Id = SPLinkDownAlert , Alerting Resource = SP Config" monitor="controller" alert_id="SPLinkDownAlert" alerting_resource="SP Config"
Mon Dec 18 19:43:11 +0000 [Node: nphmd: hm.alert.raised:alert]: Alert Id = SPAutoUpgradeFailedMajorAlert , Alerting Resource = SP Upgrade raised by monitor controller
- Rebooting SP due to low memory from SP event log:
Record 848: Sat May 29 05:47:21 2021 [IPMI.critical]: Rebooting SP due to low memory
Record 849: Thu Jan 1 00:00:46 1970 [IPMI.notice]: 7a00 | c0 | OEM: ffff70005100 | ManufId: 150300 | SP Reset Internally