NFS datastores are inaccessible due to MTU mismatch
Applies to
- NFS
- ESXi
- MTU
- ONTAP 9
Issue
- NFS datastores are mounted on ESXi servers, but attempting to read or write data hangs without error.
- Jumbo frames (MTU 9000) are configured on ESXi or ONTAP ports.
- Errors might be seen in EMS:
vifmgr.cluscheck.crcerrors: Port a0a on node NodeA is reporting a high number of observed hardware errors, possibly CRC errors.
vifmgr.cluscheck.crcerrors: Port a0a-101 on node NodeA is reporting a
high number of observed hardware errors, possibly CRC errors.
vifmgr.cluscheck.crcerrors: Port e0e on node NodeA is reporting a high number of observed hardware errors, possibly CRC errors.
vifmgr.cluscheck.crcerrors: Port e0g on node NodeA is reporting a high number of observed hardware errors, possibly CRC errors.
- New VMs can be created / storage vMotion VMs might be successful, but after data transfer there is a timeout.
- vmkernel.log: APD state is maintained intermittently.
2021-09-28T16:12:33.376Z
cpu0:2098712)WARNING: NFS: 337: Lost connection to the server
172.27.143.110 mount point /netapp_test, mounted as
8b08cbd1-435c57cd-0000-000000000000 ("netapp_test")
2021-09-28T16:14:15.781Z cpu0:2098712)NFS: 346: Restored connection to
the server 172.27.143.110 mount point /netapp_test, mounted as
8b08cbd1-435c57cd-0000-000000000000 ("netapp_test")
2021-09-28T16:14:15.781Z cpu6:2097603)StorageApdHandler: 507: APD exit event for 0x4314ef9fec20 [8b08cbd1-435c57cd]
2021-09-28T16:14:15.781Z cpu6:2097603)StorageApdHandlerEv: 117: Device
or filesystem with identifier [8b08cbd1-435c57cd] has exited the All
Paths Down state.
-
- Logs will state connection fails when trying to mount new datastores.
2024-05-15T14:20:05.123Z cpu2:2098122)NFS: 157: Unable to query remote mount point's attributes: No connection2024-05-15T14:20:10.456Z cpu2:2098122)WARNING: NFS: 1572: Failed to get attributes (No connection)
- VMware support is engaged and finds nothing wrong, stating the problem is the storage device is not responding.
Ping from ESXi succeeds, but fails with large packet size.
# 1. SUCCESS: Standard 64-byte packet (MTU 1500)[root@esxi:~] vmkping -I vmk1 10.10.10.11PING 10.10.10.11 (10.10.10.11): 64 data bytes64 bytes from 10.10.10.11: icmp_seq=0 ttl=64 time=0.421 ms64 bytes from 10.10.10.11: icmp_seq=1 ttl=64 time=0.385 ms
--- 10.10.10.11 ping statistics ---2 packets transmitted, 2 packets received, 0% packet lossround-trip min/avg/max = 0.385/0.403/0.421 ms
[root@esxi:~] vmkping -I vmk1 -s 8972 -d 10.10.10.11PING 10.10.10.11 (10.10.10.11): 8972 data bytessendto() failed (Message too long)sendto() failed (Message too long)sendto() failed (Message too long)
--- 10.10.10.11 ping statistics ---3 packets transmitted, 0 packets received, 100% packet loss
