The shelf firmware is repeatedly updating after upgrade to ONTAP 9.12.1 or 9.13.1

• ONTAP Upgrade to:
  • 9.13.1 
  • 9.12.1P4 
• Automatic background shelf firmware update
  • sfu.firmware
• SCSI Enclosure Services (SES)

• Following an ONTAP upgrade, the shelf firmware is repeatedly attempting to update. EMS shows the following events:

[node-02: dsa_disc: sfu.firmwareDownrev.shelf:error]: Shelf 0a.shelf0 has downrev firmware.
[node-02: dsa_disc: sfu.firmwareDownrev.shelf:error]: Shelf 0a.shelf1 has downrev firmware.
[node-02: dsa_sfu: sfu.firmwareDownrev:error]: Disk shelf firmware needs to be updated on 2 disk shelves.
[node-02: dsa_sfu: sfu.downloadStarted:info]: Update of disk shelf firmware started on 2 shelves.
[node-02: dsa_worker1: sfu.ctrllerElmntsPerShelf:info]: [storage download shelf]: 2 ES controller elements can be updated on 0b.shelf0.
[node-02: dsa_worker1: sfu.ctrllerElmntsPerShelf:info]: [storage download shelf]: 2 ES controller elements can be updated on 0b.shelf1.
[node-02: dsa_worker1: sfu.downloadingController:info]: [storage download shelf]: Downloading IOM12E.0250.SFW on disk shelf controller module A on 0b.shelf0.
[node-02: dsa_worker1: sfu.downloadingController:info]: [storage download shelf]: Downloading IOM12A.0310.SFW on disk shelf controller module A on 0b.shelf1.
[node-02: dsa_sfu: sfu.downloadingController:info]: [storage download shelf]: Downloading IOM12E.0250.SFW on disk shelf controller module B on 0a.shelf0.
[node-02: dsa_sfu: sfu.downloadingController:info]: [storage download shelf]: Downloading IOM12A.0310.SFW on disk shelf controller module B on 0a.shelf1.
[node-02: dsa_sfu: sfu.rebootRequest:info]: Issuing a request to reboot disk shelf 0a.shelf0 module B.
[node-02: dsa_sfu: sfu.rebootRequest:info]: Issuing a request to reboot disk shelf 0a.shelf1 module B.
[node-02: dsa_sfu: sfu.downloadSuccess:info]: [storage download shelf]: Firmware file IOM12A.0310.SFW downloaded on 0a.shelf1.
[node-02: dsa_sfu: sfu.downloadSuccess:info]: [storage download shelf]: Firmware file IOM12E.0250.SFW downloaded on 0a.shelf0.
[node-02: dsa_sfu: sfu.downloadSummary:info]: Shelf firmware updated on 2 shelves.
[node-02: storlog_admin: sla.shelf.message:debug]: params: {'type': 'SEVERITY', 'log': 'Thu Jan 1 00:00:00 1970 ( 0+00:00:00.501); 02000093; U?; HAL; hal; 04; Module Reboot: Startup type 3-Internal software reset'}
[node-02: storlog_admin: sla.shelf.mod.reboot:notice]: Reboot event reported by module A in shelf: 0b.00.99.0, log: (...) 02000093; U?; HAL; hal; 04; Module Reboot: Startup type 3-Internal software reset
[node-02: storlog_admin: sla.shelf.message:debug]: params: {'type': 'SEVERITY', 'log': (...) 02000093; U?; HAL; hal; 04; Module Reboot: Startup type 3-Internal software reset'}
[node-02: storlog_admin: sla.shelf.mod.reboot:notice]: Reboot event reported by module A in shelf: 0b.02.99.2, log: (...) 02000093; U?; HAL; hal; 04; Module Reboot: Startup type 3-Internal software reset
[node-02: storlog_admin: sla.shelf.mod.reboot:notice]: Reboot event reported by module A in shelf: 0b.03.99.3, log: (...) 02000093; U?; HAL; hal; 04; Module Reboot: Startup type 3-Internal software reset
[node-02: dsa_disc: sfu.firmwareDownrev.shelf:error]: Shelf 0a.shelf0 has downrev firmware.
[node-02: dsa_disc: sfu.firmwareDownrev.shelf:error]: Shelf 0a.shelf1 has downrev firmware.

• EMS also report various ses.status alerts

[node-02: dsa_worker5: ses.status.electronicsWarn:error]: DS212-12 shelf 0 on channel 0a environmental monitoring warning for SES electronics 2: not installed. This module is on the rear of the shelf at the top right.
[node-02:dsa_worker2: ses.status.fanWarning:debug]: DS224-12 shelf 0 on channel 0b cooling fan warning for Cooling element 1: not operating; fan is off. This module is on the rear of the shelf on the lower left power supply.
[node-02:dsa_worker2: ses.status.fanWarning:debug]: DS224-12 (S/N SHJGD2210900215) shelf 0 on channel 0b cooling fan warning for Cooling element 2: not operating; fan is off. This module is on the rear of the shelf on the lower left power supply.
[node-02:dsa_worker2: ses.status.fanWarning:debug]: DS224-12 (S/N SHJGD2210900215) shelf 0 on channel 0b cooling fan warning for Cooling element 3: not operating; fan is off. This module is on the rear of the shelf on the lower right power supply.
[node-02:dsa_worker2: ses.status.fanWarning:debug]: DS224-12 (S/N SHJGD2210900215) shelf 0 on channel 0b cooling fan warning for Cooling element 4: not operating; fan is off. This module is on the rear of the shelf on the lower right power supply.
[node-02: dsa_worker4: callhome.shlf.fan.warn:debug]: Call home for SHELF COOLING UNIT FAILED

• The shelf firmware update status might report running/idle on each node.

::> storage shelf firmware show-update-status
             Update  In-Progress
Node               Status        Count
------------------ ------- -----------
Node-02            running           4
Node-01            idle              -
2 entries were displayed.