Reocurrent Volume Latency and IOPS breaching critical threshold
Applies to
- ONTAP 9
- CLOUD VOLUMES ONTAP (CVO)
Issue
- Daily reoccurring latencies breaking 200ms/op threshold at about the same hour
- The events typically last around 10 minutes and impact a single NFS volume
- The specific client workload details and timing patterns are still pending from the customer
- Aggregate is experiencing a long CP during the same time
- qos shows Network latency as highest for the affected volume
::> qos statistics volume latency show -vserver <svm> -volume vol01Workload ID Latency Network Cluster Data Disk QoS Max QoS Min NVRAM Cloud FlexCache SM Sync VA AVSCAN--------------- ------ ---------- ---------- ---------- ---------- ---------- ---------- ---------- ---------- ---------- ---------- ---------- ---------- -----------total- - 609.56ms 523.19ms 0ms 78.13ms 187.00us 0ms 0ms 7.98ms 78.00us 0ms 0ms 0ms 0msvol01-w.. 12345 1040.89ms 918.04ms 0ms 109.95ms 169.00us 0ms 0ms 12.60ms 139.00us 0ms 0ms 0ms 0ms-total- - 460.17ms 409.43ms 0ms 45.45ms 313.00us 0ms 0ms 4.87ms 100.00us 0ms 0ms 0ms 0msvol01-w.. 12345 860.43ms 777.27ms 0ms 75.24ms 225.00us 0ms 0ms 7.50ms 191.00us 0ms 0ms 0ms 0ms-total- - 352.25ms 282.18ms 0ms 62.86ms 159.00us 0ms 0ms 6.98ms 75.00us 0ms 0ms 0ms 0msvol01-w.. 12345 950.57ms 785.62ms 0ms 148.29ms 148.00us 0ms 0ms 16.31ms 209.00us 0ms 0ms 0ms 0ms