SP firmware autoupdate fails - SP/BMC is not reachable due to API Services
Applies to
- Active IQ risk
- ONTAP 9
- Service Processor (SP)
- Baseboard Management Controller (BMC)
- SP API Service
- FAS2820
- AFF A900, ASA A900, FAS9500
- AFF A320
- AFF A220, AFF A150, ASA A150, AFF C190, FAS2720, FAS2750
- AFF A800, AFF C800, ASA A800, ASA C800
- AFF A700, FAS9000
- AFF A200, AFF A300, FAS8200, FAS2620, FAS2650
Issue
- Automated SP/BMC firmware fails to update during ONTAP upgrade:
[Node-01: servprocd: sp.servprocd.upd.evts:debug]: params: {'reason': 'BMC update - Pre-update checks successful.'}
[Node-01: servprocd: sp.servprocd.upd.evts:debug]: params: {'reason': 'SP Firmware network update from 13.7 to 13.8 has been triggered.'}
[Node-01: servprocd: sp.servprocd.upd.unexpt.evts:debug]: params: {'reason': 'BMC update - Update has timed out. Last status code (1)'}
[Node-01: servprocd: sp.servprocd.upd.error:error]: SP update error: SP firmware update failure has been detected.
- The
system health subsystem showoutput shows the Service-Processor as degraded:
::> system health subsystem show
Subsystem
Health
----------------- ------------------
SAS-connect ok
Environment ok
Memory ok
Service-Processor degraded
Switch-Health ok
CIFS-NDO ok
Motherboard ok
IO ok
MetroCluster ok
MetroCluster_Node ok
FHM-Switch ok
FHM-Bridge ok
SAS-connect_Cluster ok
13 entries were displayed.- The
SP-MGMT-MLOG-TXT.GZin the autosupport shows the following events:
[kern_servprocd:info:5478] 0x80acc8000: 0: NOTICE: Servprocd::SpUpdate: SpUpdateStateHandleEvent: SP update from version 11.7 to version 11.8 has been scheduled.
[kern_servprocd:info:5478] 0x80acc8000: 0: NOTICE: Servprocd::SpUpdate: UpdateSmfState: processing ck_auto_update_scheduled.
[kern_servprocd:info:5478] 0x80c3af200: 0: NOTICE: Servprocd::SpUpdate: prepareImage: SP firmware image files were successfully checked for compression.
[kern_servprocd:info:5478] 0x80c3af200: 0: NOTICE: Servprocd::SpUpdate: prepareImage: Creating /mroot/etc/RLM_FW/sp_image.tar.gz image for network transfer.
[kern_servprocd:info:5478] 0x80c3af200: 0: NOTICE: Servprocd::SpUpdate: transferImage: sending /mroot/etc/RLM_FW/sp_image.tar.gz to SP, destination: /mnt/sapps/sp_image.tar.gz.
[kern_servprocd:info:5478] 0x80c3af200: 0: NOTICE: Servprocd::SpUpdate: update: SP firmware image has been successfully transferred to SP using network interface.
[kern_servprocd:info:5478] 0x80c3af200: 0: NOTICE: Servprocd::SpUpdate: writeImage: sending spcs update command to write SP firmware image to flash.
[kern_servprocd:info:5478] 0x80c3af200: 0: ERR: Servprocd::SpUpdate: writeImage: spcs sp update command failed (The Service Processor on node "Node-01" is not reachable. Verify that the SP or BMC is online, verify that api-service is enabled on the SP or BMC, verify that the partner node is running, check if pings from SP or BMC to partner node work, check if hw-assist keep-alives are normal, check that network ports are configured correctly and are functional (up). Then, try the command again.)