Skip to main content
NetApp Knowledge Base

Incorrect HDD IO latency calculation leads to disk failure

  1. Last updated
  2. Save as PDF
Views:
1,636
Visibility:
Public
Votes:
1
Category:
fas-systems
Specialty:
perf
Last Updated:

Applies to

  • ONTAP 9.6-9.6P18, or versions less than 9.7P17, 9.8P9, 9.9.1P5, or 9.10.1 which do not have the fix for BUG 1383543.
  • FAS platforms

Issue

  • Multiple disk failure during a short term in one or multiple RAID groups. 
  • EMS reports disk latency has exceeded with average, so ONTAP recommends for failure:

Wed Nov 10 01:33:37 +0900 [node_name: scsi_cmdblk_strthr_admin: scsi.cmd.checkCondition:error]: Disk device 0b.00.6: Check Condition: CDB 0x8a:00000006aad4c800:00000200: Sense Data SCSI:aborted command - (0xb - 0x4b 0x6 0x0)(2031).
Wed Nov 10 01:33:37 +0900 [node_name: scsi_cmdblk_strthr_admin: scsi.cmd.retrySuccess:debug]: Disk device 0b.00.6: request successful after retry #1/#0: cdb 0x8a:00000006aad4c800:00000200 (2147).
Wed Nov 10 01:33:48 +0900 [node_name: disk_latency_monitor: shm.threshold.ioLatency:debug]: Disk 0b.00.6 has exceeded the expected IO latency in the current window with average latency of 15122 msecs and average utilization of 47 percent. Highest average IO latency: 0b.00.6: 15122 msecs; next highest IO latency: 0b.00.10: 11 msecs. Disk 0b.00.6 Shelf 0 Bay 6 [NETAPP X388_SEVNE16TA07 NA00] S/N [ZL25117M] UID [5000C500:CA04E0EF:00000000:00000000:00000000:00000000:00000000:00000000:00000000:00000000]
Wed Nov 10 01:34:18 +0900 [node_name: disk_latency_monitor: shm.threshold.ioLatency:debug]: Disk 0b.00.6 has exceeded the expected IO latency in the current window with average latency of 15037 msecs and average utilization of 47 percent. Highest average IO latency: 0b.00.6: 15037 msecs; next highest IO latency: 0b.00.10: 11 msecs. Disk 0b.00.6 Shelf 0 Bay 6 [NETAPP X388_SEVNE16TA07 NA00] S/N [ZL25117M] UID [5000C500:CA04E0EF:00000000:00000000:00000000:00000000:00000000:00000000:00000000:00000000]
Wed Nov 10 01:34:48 +0900 [node_name: disk_latency_monitor: shm.threshold.ioLatency:debug]: Disk 0b.00.6 has exceeded the expected IO latency in the current window with average latency of 14792 msecs and average utilization of 50 percent. Highest average IO latency: 0b.00.6: 14792 msecs; next highest IO latency: 0b.00.10: 11 msecs. Disk 0b.00.6 Shelf 0 Bay 6 [NETAPP X388_SEVNE16TA07 NA00] S/N [ZL25117M] UID [5000C500:CA04E0EF:00000000:00000000:00000000:00000000:00000000:00000000:00000000:00000000]
Wed Nov 10 01:34:48 +0900 [node_name: disk_latency_monitor: shm.threshold.highIOLatency:error]: Disk 0b.00.6 exceeds the average IO latency threshold and will be recommended for failure.
Wed Nov 10 01:34:48 +0900 [node_name: disk_latency_monitor: scsi.debug:debug]: shm_setup_for_failure disk 0b.00.6 (S/N ZL25117M) error 200000h

 

 

Sign in to view the entire content of this KB article.

New to NetApp?

Learn more about our award-winning Support

NetApp provides no representations or warranties regarding the accuracy or reliability or serviceability of any information or recommendations provided in this publication or with respect to any results that may be obtained by the use of the information or observance of any recommendations provided herein. The information in this document is distributed AS IS and the use of this information or the implementation of any recommendations or techniques herein is a customer's responsibility and depends on the customer's ability to evaluate and integrate them into the customer's operational environment. This document and the information contained herein may be used solely in connection with the NetApp products discussed in this document.