Skip to main content

NetApp wins prestigious Coveo Relevance Pinnacle Award. Learn more!

INSIGHT Japan :2023年 1月25日(水)ANAインターコンチネンタルホテル開催 へ参加・申込を行う

NetApp Knowledge Base

Volume manual reconstruction failure after vdmRecoverAllRAIDVols following power loss

Views:
155
Visibility:
Public
Votes:
0
Category:
e-series-santricity-os-controller-software
Specialty:
esg
Last Updated:

Applies to

E-series

Issue

  • Volume failure after power loss
  • Volume reconstruction not starts after running vdmRecoverAllRAIDVols
  • Volume manual reconstruction is started manually but fail:
  • Fail drives which have failed pieces manually (on System Manager > Hardware > click the drive > Fail). Major Event Log sample:

A:8/4/22, 3:29:43 AM (03:29:43) 8583 2226 Drive spun down - Shelf 0, Drawer 4, Bay 8
B:8/4/22, 3:29:44 AM (03:29:44) 8582 2226 Drive spun down - Shelf 0, Drawer 4, Bay 8
B:8/4/22, 3:29:43 AM (03:29:43) 8581 6008 Stable storage drive unusable - Shelf 0, Drawer 4, Bay 8
A:8/4/22, 3:29:43 AM (03:29:43) 8580 5023 Controller return status/function call for requested operation - Shelf 99, Bay A
----> RPC function : 0x000b = setDriveToFailed
----> Return code  : 0x0001 = ok
A:8/4/22, 3:29:43 AM (03:29:43) 8579 222d Drive manually failed - Shelf 0, Drawer 4, Bay 8 <--CRITICAL
A:8/4/22, 3:29:43 AM (03:29:43) 8578 5006 Fail drive - Shelf 0, Drawer 4, Bay 8
A:8/4/22, 3:29:43 AM (03:29:43) 8577 2215 Drive marked failed - Shelf 0, Drawer 4, Bay 8
A:8/4/22, 3:29:43 AM (03:29:43) 8576 226c Drive failure - Shelf 0, Drawer 4, Bay 8 - Cause: 2 = User failed; Drive WWN: ; SN:  <--CRITICAL
A:8/4/22, 3:29:42 AM (03:29:42) 8575 2226 Drive spun down - Shelf 0, Drawer 4, Bay 8
A:8/4/22, 3:29:42 AM (03:29:42) 8574 6008 Stable storage drive unusable - Shelf 0, Drawer 4, Bay 8
B:8/4/22, 3:29:25 AM (03:29:25) 8573 2226 Drive spun down - Shelf 0, Drawer 3, Bay 10
B:8/4/22, 3:29:24 AM (03:29:24) 8572 5023 Controller return status/function call for requested operation - Shelf 99, Bay B
----> RPC function : 0x000b = setDriveToFailed
----> Return code  : 0x0001 = ok
B:8/4/22, 3:29:24 AM (03:29:24) 8571 222d Drive manually failed - Shelf 0, Drawer 3, Bay 10 <--CRITICAL
B:8/4/22, 3:29:24 AM (03:29:24) 8570 5006 Fail drive - Shelf 0, Drawer 3, Bay 10
A:8/4/22, 3:29:23 AM (03:29:23) 8569 2226 Drive spun down - Shelf 0, Drawer 3, Bay 10
A:8/4/22, 3:29:23 AM (03:29:23) 8568 6008 Stable storage drive unusable - Shelf 0, Drawer 3, Bay 10
B:8/4/22, 3:29:24 AM (03:29:24) 8567 2215 Drive marked failed - Shelf 0, Drawer 3, Bay 10
B:8/4/22, 3:29:24 AM (03:29:24) 8566 226c Drive failure - Shelf 0, Drawer 3, Bay 10 - Cause: 2 = User failed; Drive WWN: ; SN:  <--CRITICAL
B:8/4/22, 3:29:24 AM (03:29:24) 8565 2226 Drive spun down - Shelf 0, Drawer 3, Bay 10
B:8/4/22, 3:29:24 AM (03:29:24) 8564 6008 Stable storage drive unusable - Shelf 0, Drawer 3, Bay 10

  • Reconstruct those failed drives (Hardware > click the drive > Reconstruct). Major Event Log sample:

A:8/4/22, 3:30:58 AM (03:30:58) 8597 6009 Stable storage drive usable - Shelf 0, Drawer 3, Bay 10
A:8/4/22, 3:30:47 AM (03:30:47) 8596 5023 Controller return status/function call for requested operation - Shelf 99, Bay A
----> RPC function : 0x0013 = startDriveReconstruction
----> Return code  : 0x0001 = ok
A:8/4/22, 3:30:47 AM (03:30:47) 8595 500e Reconstruct drive/volume - Shelf 0, Drawer 4, Bay 8
A:8/4/22, 3:30:42 AM (03:30:42) 8594 2227 Drive marked optimal - Shelf 0, Drawer 4, Bay 8
B:8/4/22, 3:31:24 AM (03:31:24) 8593 6008 Stable storage drive unusable - Shelf 0, Drawer 3, Bay 3
B:8/4/22, 3:30:59 AM (03:30:59) 8592 6009 Stable storage drive usable - Shelf 0, Drawer 3, Bay 10
B:8/4/22, 3:30:48 AM (03:30:48) 8591 2027 Reconstruction resumed - Volume Vol02
B:8/4/22, 3:30:45 AM (03:30:45) 8590 2026 Reconstruction completed - Volume Vol02
B:8/4/22, 3:30:18 AM (03:30:18) 8589 2025 Reconstruction started - Volume Vol02
B:8/4/22, 3:30:18 AM (03:30:18) 8588 2224 Reconstruction started - Shelf 0, Drawer 3, Bay 10
B:8/4/22, 3:30:07 AM (03:30:07) 8587 5023 Controller return status/function call for requested operation - Shelf 99, Bay B
----> RPC function : 0x0013 = startDriveReconstruction
----> Return code  : 0x0001 = ok
B:8/4/22, 3:30:07 AM (03:30:07) 8586 500e Reconstruct drive/volume - Shelf 0, Drawer 3, Bay 10
B:8/4/22, 3:30:06 AM (03:30:06) 8585 2023 Media scan (scrub) completed - Volume Vol02
B:8/4/22, 3:30:05 AM (03:30:05) 8584 2227 Drive marked optimal - Shelf 0, Drawer 3, Bay 10

  • A drive fail thus reconsturction fail: 

B:8/4/22, 9:08:37 AM (09:08:37) 8964 2215 Drive marked failed - Shelf 0, Drawer 4, Bay 8
B:8/4/22, 9:08:37 AM (09:08:37) 8963 226c Drive failure - Shelf 0, Drawer 4, Bay 8 - Cause: 3 = Write failure; Drive WWN: ; SN:  <--CRITICAL
B:8/4/22, 9:08:36 AM (09:08:36) 8962 2226 Drive spun down - Shelf 0, Drawer 4, Bay 8
B:8/4/22, 9:08:36 AM (09:08:36) 8961 6008 Stable storage drive unusable - Shelf 0, Drawer 4, Bay 8
B:8/4/22, 9:08:28 AM (09:08:28) 8960 6703 Overflow in unreadable sector database - Volume Vol02 <--CRITICAL
B:8/4/22, 9:08:28 AM (09:08:28) 8959 6700 Unreadable sector(s) detected data loss occurred - Volume Vol02 - LBA: 0x51dc82ca0 <--CRITICAL
----> Physical Drive in Tray 0 Slot 35, LBA: 0xa3b904a0
B:8/4/22, 9:08:28 AM (09:08:28) 8958 2061 Data assurance mismatch detected - probable cause is cached data - Volume Vol02 - ioType: DST_OUT, hwPIStatus: GUARD_ERROR, swPIStatus: GUARD_ERROR, Host ID: 65535, LBA: 0x51dc82c00
----> Expected Guard:0x0000, AppTag:0x0000, RefTag:0x00000000
----> Found    Guard:0x0000, AppTag:0x0000, RefTag:0x00000000
B:8/4/22, 9:08:28 AM (09:08:28) 8957 2061 Data assurance mismatch detected - probable cause is cached data - Volume Vol02 - ioType: DST_OUT, hwPIStatus: GUARD_ERROR, swPIStatus: GUARD_ERROR, Host ID: 65535, LBA: 0x51dc83000
----> Expected Guard:0x0000, AppTag:0x0000, RefTag:0x00000000
----> Found    Guard:0x0000, AppTag:0x0000, RefTag:0x00000000
B:8/4/22, 9:08:28 AM (09:08:28) 8956 1012 Destination driver error - Shelf 0, Drawer 4, Bay 8
----> Fail Reason: Last Error, Edc: 0x2-0/0/0 - Channel 1
----> Error#1: Edc: 0x2-0/0/0, Ch:3, Next:Retry only on other ITNexus; Error#2: Edc: 0x2-0/0/0, Ch:1, Next:Fail command;

 

Sign in to view the entire content of this KB article.

New to NetApp?

Learn more about our award-winning Support

Scan to view the article on your device