High latency and disk errors with single/few disk high utilization
Applies to
- ONTAP 9
- FAS systems
Issue
- Slowness reported by multiple users and high disk utilization reported.
- Multiple alerts/emails received reporting high latency from various volumes :
-
7476 Sep 20 08:19:24 hdisk68 P SC_DISK_ERR7 path 0 path failure; READ(16) (0000000102206B,00000200) command timeout 7476 Sep 20 08:19:24 hdisk68 P SC_DISK_ERR7 path 0 path failure; READ(16) (0000000102206B,00000200) command timeout Oct 5 08:21:42 Message forwarded from fb-lp-tsm01: root: ERRPT: DE3B8540 1005082122 P H hdisk53 PATH HAS FAILED
- statit command report high utilization on single disk.
Cluster::*> node run -node node1 -command statit -e
...
disk ut% xfers ureads--chain-usecs writes--chain-usecs cpreads-chain-usecs greads--chain-usecs gwrites-chain-usecs
/aggr1_node1/plex0/rg2:
0d.23.18 25 69.19 0.01 2.00 2033 31.50 60.67 175 37.69 51.19 169 0.00 .... . 0.00 .... .
0a.21.16 24 69.77 0.01 2.00 6964 32.11 59.56 194 37.66 51.18 90 0.00 .... . 0.00 .... .
0d.22.17 57 231.26 133.56 5.45 2075 26.91 29.41 573 70.79 9.87 641 0.00 .... . 0.00 .... .
0d.23.22 57 230.68 132.96 5.46 1845 26.83 29.56 646 70.90 9.74 604 0.00 .... . 0.00 .... .
0d.23.13 95 295.63 198.16 4.10 5472 26.83 29.76 1371 70.63 9.91 1975 0.01 .... . 0.00 .... .
0d.22.18 57 231.26 133.55 5.38 2080 26.84 29.60 561 70.86 9.73 634 0.00 .... . 0.00 .... .
0a.20.18 57 231.69 133.54 5.42 1846 27.00 29.46 647 71.15 9.78 608 0.00 .... . 0.00 .... .
0a.20.16 57 233.00 134.49 5.48 1879 27.08 30.09 634 71.43 9.83 594 0.00 .... . 0.00 .... .
0d.22.19 57 231.98 134.18 5.41 2099 26.87 29.67 567 70.93 9.87 646 0.00 .... . 0.00 .... .