Esx device timeout due to high latency
Applies to
- ONTAP 9
- FCP
Issue
- Alerts like below are seen in Oncommand
The device latency in microseconds exceeded the threshold value for device nvd0 and the associated target device n/a. The threshold is 25000, execution latency is 27444, total latency is 27448, and the number of operations for this period is 32.
- Esxi servers timeout and get disconnected
- There is high latency seen as per below QOS command output for the impacted volumes
::> qos statistics volume latency show workload wid latency nblade cluster dblade disk qos_limit qos_floor nvlog cloud flexcache sync_repl cop avscan Workload ID Latency Network Cluster Data Disk QoS Max QoS Min NVRAM Cloud FlexCache SM Sync VA AVSCAN -total- - 12.22ms 143.00us 11.95ms 107.00us 2.00us 0ms 0ms 15.00us 0ms 0ms 0ms 0ms 0ms Volume1 410.54ms 73.00us 410.45ms 15.00us 0ms 0ms 0ms 2.00us 0ms 0ms 0ms 0ms 0ms Volume2 384.68ms 66.00us 384.60ms 13.00us 0ms 0ms 0ms 1.00us 0ms 0ms 0ms 0ms 0ms Volume3 272.12ms 70.00us 272.02ms 23.00us 0ms 0ms 0ms 2.00us 0ms 0ms 0ms 0ms 0ms Volume4 266.23ms 87.00us 266.05ms 86.00us 0ms 0ms 0ms 4.00us 0ms 0ms 0ms 0ms 0ms