High read latency due to high disk utilization

Last updated
Save as PDF
Share
1. Share
2. Tweet
3. Share

Views:: 514

Visibility:: Public

Votes:: 0

Category:: ontap-9

Specialty:: Perf

Last Updated:

Applies to

ONTAP 9

Issue

Performance degradation is experienced on the client
Active IQ Unified Manager (AIQUM) monitoring the cluster raises an alert of higher latency

Example: Latency value of XX.X ms/op on volname has triggered a WARNING event based on threshold setting of 20.0 ms/op.

The read latency is 25~40ms
Disk utilization is high according to sysstat -x

CPU NFS CIFS HTTP Total Net kB/s HDD kB/s SSD kB/s Tape kB/s Cache Cache CP CP_Ty CP_Ph HDD SSD OTHER FCP iSCSI FCP kB/s in out read write read write read write age hit time [T--H--F--N--B--O--#--:] [n--v--p--f] util util in out 13% 0 0 0 3724 4 10 77324 1612 0 0 0 0 3s 96% 15% 0--0--0--0--0--0--0--0 0--0--0--0 76% 0% 4 3720 0 6578 161769 9% 0 0 0 2968 12 9 63968 0 0 0 0 0 4s 96% 0% 0--0--0--0--0--0--0--0 0--0--0--0 79% 0% 6 2962 0 4322 144081 13% 0 0 0 3256 5 8 40576 36 0 0 0 0 1s 96% 0% 0--0--0--0--0--0--0--0 0--0--0--0 90% 0% 3 3253 0 2777 155900 11% 0 0 0 3362 5 6 46656 0 0 0 0 0 4s 96% 0% 0--0--0--0--0--0--0--0 0--0--0--0 80% 0% 203 3159 0 1909 152156 14% 0 0 0 3216 11 12 61876 0 0 0 0 0 2s 97% 0% 0--0--0--0--0--0--0--0 0--0--0--0 86% 0% 16 3200 0 7192 138334 10% 0 0 0 1501 315 58 47940 25704 0 0 0 0 2s 97% 34% 1--0--0--0--0--0--0--0 0--0--0--1 90% 0% 5 1496 0 2362 64046 9% 0 0 0 1549 3 3 32836 46760 0 0 0 0 4s 96% 100% 0--0--0--0--0--0--0--1 0--0--0--1 100% 0% 422 1127 0 655 51515 7% 0 0 0 1751 13 15 45812 8564 0 0 0 0 5s 96% 61% 0--0--0--0--0--0--0--0 0--0--0--0 100% 0% 11 1740 0 983 86513 10% 0 0 0 2513 6 8 48896 0 0 0 0 0 6s 96% 0% 0--0--0--0--0--0--0--0 0--0--0--0 97% 0% 9 2504 0 3554 125773 9% 0 0 0 2473 7 8 42328 0 0 0 0 0 7s 97% 0% 0--0--0--0--0--0--0--0 0--0--0--0 88% 0% 5 2468 0 1208 131218 9% 0 0 0 2253 13 11 48852 1832 0 0 0 0 1s 96% 39% 0--0--0--0--0--0--0--0 0--0--0--0 91% 0% 188 2065 0 2110 107723 8% 0 0 0 2217 4 6 52248 0 0 0 0 0 6s 96% 0% 0--0--0--0--0--0--0--0 0--0--0--0 99% 0% 3 2214 0 1327 106689 9% 0 0 0 2060 6 6 60216 36 0 0 0 0 1s 97% 0% 0--0--0--0--0--0--0--0 0--0--0--0 88% 0% 4 2056 0 598 119959 8% 0 0 0 2254 5 7 58528 0 0 0 0 0 2s 96% 0% 0--0--0--0--0--0--0--0 0--0--0--0 95% 0% 205 2049 0 1991 108034 13% 0 0 0 4376 10 10 177960 0 0 0 0 0 2s 94% 0% 0--0--0--0--0--0--0--0 0--0--0--0 79% 0% 8 4368 0 1174 252326 16% 0 0 0 5209 7 10 221796 24 0 0 0 0 3s 95% 0% 0--0--0--0--0--0--0--0 0--0--0--0 69% 0% 2 5207 0 531 318983