Two disks partitioned simultaneously and displaying "SpareLow" during disk failure

Last updated

Aug 30, 2024
Save as PDF
Share
1. Share
2. Tweet
3. Share

Views:: 104

Visibility:: Public

Votes:: 0

Category:: ontap-9

Specialty:: CORE

Last Updated:: 8/30/2024, 11:53:14 AM

Applies to

FAS2620
9.9.1P15

Issue

During a disk failure, two disks are partitioned simultaneously, resulting in the "SpareLow" condition.

disk failed.

Mon Apr 01 14:46:34 +0900 [node-01: config_thread: callhome.fdsk.fault:error]: Call home for FILESYSTEM DISK FAILED Shelf 10, Bay 9, Model [X380_HLBRE10TA07], S/N [XXXXXXXX]

Disk partition on the first spare disk.

[?] Mon Apr 01 14:46:35 +0900 [node-01: config_thread: raid.partition.disk:notice]: Disk partition successful on Disk 0a.10.2 Shelf 10 Bay 2 [NETAPP X380_STATE10TA07 NA00] S/N [XXXXXXXX] UID [5000C500:A75AC483:00000000:00000000:00000000:00000000:00000000:00000000:00000000:00000000], partitions created 2, partition sizes specified 1, partition spec summary [1]=2372885888. [?] Mon Apr 01 14:46:36 +0900 [node-01: disk_layout_admin: disk.layout.changed:debug]: Detected partition layout change on disk 0a.10.2.

[?] Mon Apr 01 14:46:36 +0900 [node-01: dmgr_thread: raid.disk.inserted:info]: Disk 0a.10.2 Shelf 10 Bay 2 [NETAPP X380_STATE10TA07 NA00] S/N [XXXXXXXX] UID [5000C500:A75AC483:00000000:00000000:00000000:00000000:00000000:00000000:00000000:00000000] has been inserted into the system

After partitioning spare disk, this disk is missing on sysconfig -r.

Pool0 spare disks

RAID Disk Device HA SHELF BAY CHAN Pool Type RPM Used (MB/blks) Phys (MB/blks) --------- ------ ------------- ---- ---- ---- ----- -------------- -------------- Spare disks for block checksum spare 0a.10.3P2 0a 10 3 SA:B 0 FSAS 7200 55176/113000448 55184/113016832 spare 0a.10.4P2 0a 10 4 SA:B 0 FSAS 7200 55176/113000448 55184/113016832 spare 0a.10.6P2 0a 10 6 SA:B 0 FSAS 7200 55176/113000448 55184/113016832 spare 0a.10.7P2 0a 10 7 SA:B 0 FSAS 7200 55176/113000448 55184/113016832 spare 0a.10.8P2 0a 10 8 SA:B 0 FSAS 7200 55176/113000448 55184/113016832 spare 0a.10.10P2 0a 10 10 SA:B 0 FSAS 7200 55176/113000448 55184/113016832 (not zeroed) spare 0a.10.11P2 0a 10 11 SA:B 0 FSAS 7200 55176/113000448 55184/113016832 spare 0b.00.0P2 0b 0 0 SA:A 0 FSAS 7200 55176/113000448 55184/113016832 spare 0b.00.5P2 0b 0 5 SA:A 0 FSAS 7200 55176/113000448 55184/113016832 spare 0b.00.11P2 0b 0 11 SA:A 0 FSAS 7200 55176/113000448 55184/113016832 spare 0a.10.5 0a 10 5 SA:B 0 FSAS 7200 9324290/19096145920 9342976/19134414848 (not zeroed)

So the system start disk partition on another spare disk.

[?] Mon Apr 01 14:46:55 +0900 [node-01: config_thread: raid.partition.disk:notice]: Disk partition successful on Disk 0a.10.5 Shelf 10 Bay 5 [NETAPP X380_HLBRE10TA07 NA03] S/N [XXXXXXXX] UID [5000CCA2:5172ED2C:00000000:00000000:00000000:00000000:00000000:00000000:00000000:00000000], partitions created 2, partition sizes specified 1, partition spec summary [1]=2372885888. [?] Mon Apr 01 14:46:55 +0900 [node-01: disk_layout_admin: disk.layout.changed:debug]: Detected partition layout change on disk 0a.10.5. [?] Mon Apr 01 14:46:55 +0900 [node-01: rastrace_dump: rastrace.dump.saved:debug]: A RAS trace dump for module RAID instance 0 was stored in /etc/log/rastrace/RAID_0_20240401_14:46:55:336307.dmp. [?] Mon Apr 01 14:46:55 +0900 [node-01: dmgr_thread: raid.disk.inserted:info]: Disk 0a.10.5 Shelf 10 Bay 5 [NETAPP X380_HLBRE10TA07 NA03] S/N [XXXXXXXX] UID [5000CCA2:5172ED2C:00000000:00000000:00000000:00000000:00000000:00000000:00000000:00000000] has been inserted into the system

Trigger SPARES_LOW Autosupport message

[?] Mon Apr 01 14:46:56 +0900 [node-01: config_thread: raid.rg.spares.low:error]: /aggr01_02/plex0/rg0 [?] Mon Apr 01 14:46:56 +0900 [node-01: config_thread: callhome.spares.low:error]: Call home for SPARES_LOW: /aggr01_02/plex0/rg0

The system picked up the first partitioned spare disk for reconstruction.

[?] Mon Apr 01 14:48:21 +0900 [node-01: config_thread: raid.rg.recons.info:notice]: Spare disk 0a.10.2P1 will be used to reconstruct one missing disk in RAID group /aggr01_02/plex0/rg0.

unpartitioned the second spare disk.

[?] Mon Apr 01 14:51:13 +0900 [node-01: config_thread: raid.unpartition.disk:notice]: Disk unpartition successful on Disk 0a.10.5 Shelf 10 Bay 5 [NETAPP X380_HLBRE10TA07 NA03] S/N [XXXXXXXX] UID [5000CCA2:5172ED2C:00000000:00000000:00000000:00000000:00000000:00000000:00000000:00000000]. [?] Mon Apr 01 14:51:13 +0900 [node-01: disk_layout_admin: disk.layout.changed:debug]: Detected partition layout change on disk 0a.10.5.