Volumes fail when upgrading SANtricity OS with a failed power supply
Applies to
NetApp E-Series
Issue
Volumes fail when upgrading SANtricity OS with a failed power supply:
A:8/18/21, 7:32:35 PM (19:32:35) 4366 7e00 Drive power toggled off - Shelf 99, Bay 23
A:8/18/21, 7:32:35 PM (19:32:35) 4365 7e00 Drive power toggled off - Shelf 99, Bay 22
A:8/18/21, 7:32:35 PM (19:32:35) 4364 7e00 Drive power toggled off - Shelf 99, Bay 21
A:8/18/21, 7:32:35 PM (19:32:35) 4363 7e00 Drive power toggled off - Shelf 99, Bay 20
A:8/18/21, 7:32:35 PM (19:32:35) 4362 7e00 Drive power toggled off - Shelf 99, Bay 19
A:8/18/21, 7:32:35 PM (19:32:35) 4361 7e00 Drive power toggled off - Shelf 99, Bay 18
A:8/18/21, 7:32:35 PM (19:32:35) 4360 2250 Volume failure - Volume volume4 <--CRITICAL
A:8/18/21, 7:32:35 PM (19:32:35) 4359 2250 Volume failure - Volume volume3 <--CRITICAL
A:8/18/21, 7:32:35 PM (19:32:35) 4358 6008 Stable storage drive unusable - Shelf 99, Bay 18
A:8/18/21, 7:32:35 PM (19:32:35) 4357 6008 Stable storage drive unusable - Shelf 99, Bay 22
A:8/18/21, 7:32:35 PM (19:32:35) 4356 6008 Stable storage drive unusable - Shelf 99, Bay 20
A:8/18/21, 7:32:35 PM (19:32:35) 4355 6008 Stable storage drive unusable - Shelf 99, Bay 23
A:8/18/21, 7:32:35 PM (19:32:35) 4354 6008 Stable storage drive unusable - Shelf 99, Bay 21
A:8/18/21, 7:32:35 PM (19:32:35) 4353 6008 Stable storage drive unusable - Shelf 99, Bay 19
...
A:8/18/21, 7:32:33 PM (19:32:33) 4346 1012 Destination driver error - Shelf 99, Bay 18
----> Fail Reason: Last Error, Hid: Device fail timeout (all ITNs to device have been disconnected for too long) - Channel 2
----> Error#1: Sas: 0x2-0/e5/2, Ch:1, Next:Retry on other ITNexus; Error#2: Hid: Device fail timeout (all ITNs to device have been disconnected for too long), Ch:2, Next:Fail command;
A:8/18/21, 7:32:33 PM (19:32:33) 4345 1012 Destination driver error - Shelf 99, Bay 18
----> Fail Reason: Last Error, Hid: Device fail timeout (all ITNs to device have been disconnected for too long) - Channel 1
----> Error#1: Sas: 0x2-0/e5/2, Ch:2, Next:Retry on other ITNexus; Error#2: Hid: Device fail timeout (all ITNs to device have been disconnected for too long), Ch:1, Next:Fail command;
...
A:8/18/21, 7:32:20 PM (19:32:20) 4335 5024 Internal download checkpoint - None - Checkpoint 68: CFW_ACTV_WAITING_FOR_ALT_TO_RESET
A:8/18/21, 7:32:20 PM (19:32:20) 4334 400f Controller reset by its alternate - Shelf 99, Bay B
----> Reboot Reason : SOFD: REBOOTALT_SOFD_CONTINUE_ACTIVATION_AFTER_REBOOT
A:8/18/21, 7:32:20 PM (19:32:20) 4333 5024 Internal download checkpoint - None - Checkpoint 67: CFW_ACTV_RESETTING_ALTERNATE
...
A:8/18/21, 7:32:01 PM (19:32:01) 4311 5024 Internal download checkpoint - None - Checkpoint 66: CFW_ACTV_TAKING_VOLUMES_FROM_ALT
A:8/18/21, 7:32:01 PM (19:32:01) 4310 2605 Start-of-day routine completed - Shelf 99, Bay A
...
A:8/18/21, 7:31:54 PM (19:31:54) 4303 4010 Controller reset - Shelf 99, Bay A
----> Reboot Reason : SOFD: REBOOT_SOFD_ACTIVATED_STAGED_CTLR_FW_DOWNLOAD
A:8/18/21, 7:31:54 PM (19:31:54) 4302 283b Power supply failed - Shelf 99, Power-fan canister 2, Bay 1 <--CRITICAL
A:8/18/21, 7:31:54 PM (19:31:54) 4301 3101 Asynchronous event notification (AEN) posted for recently logged event - Shelf 99, Bay A
----> Sense 6/3f/c7 = Unit Attention - Non-media Component Failure
----> FRU group: Subsystem; Tray ID: 99, Type : Power Supply, State: Failed
A:8/18/21, 7:31:46 PM (19:31:46) 4300 2606 Start-of-day routine begun - Shelf 99, Bay A
...
A:8/18/21, 7:30:24 PM (19:30:24) 4297 5046 Activate controller firmware completed - Shelf 99, Bay A
A:8/18/21, 7:30:23 PM (19:30:23) 4296 5024 Internal download checkpoint - None - Checkpoint 64: CFW_ACTV_REBOOTING_MYSELF_DUAL_ACTIVE
...
A:8/18/21, 7:30:09 PM (19:30:09) 4291 5024 Internal download checkpoint - None - Checkpoint 61: CFW_ACTV_GIVING_VOLUMES_TO_ALTERNATE
A:8/18/21, 7:30:06 PM (19:30:06) 4290 5024 Internal download checkpoint - None - Checkpoint 60: CFW_ACTV_ACTIVATION_STARTED
A:8/18/21, 7:30:06 PM (19:30:06) 4289 5045 Activate controller firmware started - Shelf 99, Bay A
A:8/18/21, 7:28:40 PM (19:28:40) 4288 5026 Controller firmware download completed - Shelf 99, Bay A - (08.63.00.02 -> 08.71.03.00)
A:8/18/21, 7:28:40 PM (19:28:40) 4287 5028 Controller NVSRAM download completed - Shelf 99, Bay A
A:8/18/21, 7:28:40 PM (19:28:40) 4286 500b Controller NVSRAM download started - Shelf 99, Bay A
A:8/18/21, 7:27:20 PM (19:27:20) 4285 5009 Controller firmware download started - Shelf 99, Bay A - (08.63.00.02 -> 08.71.03.00)