Virtual Machines go offline with All Path Down error due to high CRC errors
Applies to
- ONTAP 9
- VMware ESXi
Issue
-
VMware logs indicate VMs goes offline with datastore entering APD (All Paths Down):
2022-11-26T06:54:46.052Z cpu84:2098519)WARNING: NFS: 338: Lost connection to the server vmfilerXX mount point /vol/vol1, mounted as XXXXXXXX-XXXXXXXX-XXXX-XXXXXXXXXXXX ("XXXXX")
2022-11-26T06:54:46.052Z cpu84:2098519)StorageApdHandler: 1191: APD start for 0x431eXXXXX3f0 [11XXXX90-4fXXXXf5]
2022-11-26T06:54:46.052Z cpu27:2097742)StorageApdHandler: 408: APD start event for 0x431eXXXXXf0 [11XXXX90-4fXXXXf5]
- Volume latency changes are reported:
2022-11-26T06:21:50.842Z cpu22:2098518)WARNING: NFS: 5015: NFS volume vol1 performance has deteriorated. I/O latency increased from average value of 6511(us) to 6905036(us).
2022-11-26T06:21:50.844Z cpu8:2098518)NFS: 4971: NFS volume vol1 performance has improved. I/O latency reduced from 6905036(us) to 1364914(us).