Aggregate offline due to shelf removal
Applies to
- FAS8200
- 9.8P19
Issue
- Aggregate offline due to shelf removal
sysconfig - rshows
*** This system has failed.
Any adapters shown below are those of the live partner, nodename
Aggregate aggr_name (failed, raid_tec, partial) (block checksums)
Plex /AGGR_name/plex0 (offline, failed, inactive)
RAID group /aggr_name/plex0/rg0 (partial, block checksums)
RAID Disk Device HA SHELF BAY CHAN Pool Type RPM Used (MB/blks) Phys (MB/blks)
--------- ------ ------------- ---- ---- ---- ----- -------------- --------------
tparity FAILED N/A 5614621/ -
dparity 0c.31.0 0c 31 0 SA:B 0 FSAS 7200 5614621/11498743808 5625872/11521787400
parity FAILED N/A 5614621/ -
data 0c.41.22 0c 41 22 SA:B 0 BSAS 7200 1695466/3472315904 1695759/3472914816
[...]
data FAILED N/A 5614621/ -
data 2d.33.0 2d 33 0 SA:A 0 FSAS 7200 5614621/11498743808 5625872/11521787400
data FAILED N/A 5614621/ -
data FAILED N/A 5614621/ -
data 2d.33.2 2d 33 2 SA:A 0 FSAS 7200 5614621/11498743808 5625872/11521787400
data FAILED N/A 5614621/ -
Raid group is missing 10 disks.
- Following events are seen in EMS
- Multiple disks report as missing
[nodename: config_thread: raid.config.filesystem.disk.missing:info]: File system Disk /aggr_name/plex0/rg0/0b.38.21 Shelf 38 Bay 21 [NETAPP X316_HAKPE06TA07 NA00] S/N [XXXXXXXXXX] UID [5000CCA2:55635228:00000000:00000000:00000000:00000000:00000000:00000000:00000000:00000000] is missing.
[nodename: config_thread: raid.config.filesystem.disk.missing:info]: File system Disk /aggr_name/plex0/rg0/0b.37.22 Shelf 37 Bay 22 [NETAPP X316_TTCRE06TA07 NA03] S/N [XXXXXXXXXX] UID [50000397:88131804:00000000:00000000:00000000:00000000:00000000:00000000:00000000:00000000] is missing.
- Raid group is then reported as degraded
[nodename: config_thread: raid.rg.degraded:notice]: : Raid group /aggr_name/plex0/rg0 is degraded- Multidisk error and no valid paths to disks
[nodename: config_thread: cf.multidisk.fatalProblem:error]: Node encountered a multidisk error or other fatal error while waiting to be taken over. aggr aggr_name : raid volfsm, fatal multi-disk error.. Raid type - raid_tec Group name plex0/rg0 state NORMAL. 6 disks failed in the group. Disk 0b.38.21 Shelf 38 Bay 21 [NETAPP X316_HAKPE06TA07 NA00] S/N [XXXXXXXXXX] UID [5000CCA2:55635228:00000000:00000000:00000000:00000000:00000000:00000000:00000000:00000000] error: disk does not exist. Disk 0b.37.21 Shelf 37 Bay 21 [NETAPP X316_TTCRE06TA07 NA03] S/N [XXXXXXXXXX] UID [50000397:881318A0:00000000:00000000:00000000:00000000:00000000:00000000:00000000:00000000] error: no valid path to disk. Disk 0b.38.22 Shelf 38 Bay 22 [NETAPP X316_HAKPE06TA07 NA00] S/N [XXXXXXXXXX] UID [5000CCA2:556543E0:00000000:00000000:00000000:00000000:00000000:00000000:00000000:00000000] error: disk does not exist. Disk 0b.37.22 Shelf 37 Bay 22 [NETAPP X316_TTCRE06TA07 NA03] S/N [XXXXXXXXXX] UID [50000397:88131804:00000000:00000000:000.