NAS access slow or hung caused by CRC errors on network port
Applies to
- ONTAP 9
- NFS
- CIFS
Issue
- Observed slow data transfer and high read latency every time application is run
- QOS statistics volume latency command slows little latency
- NFS client report below error and hang:
kernel: nfs: server 10.xx.xx.xx not responding, still trying
- Packet trace (from client or storage) may show retransmit and out of order packets
- In CIFS the shares experience slowness in reads and writes
- Check for network errors
- From the filer you can check ports with the below command
- >>:
node run -node node1 -command "ifstat-a"
for the affected ports
- >>:
- Engage network team to review connected switch ports as well
- From the filer you can check ports with the below command
-
CRC errors: 35927 | Runt frames: 0 Fragment: 0 | Long frames: 0 | Jabber: 0 Length errors: 0 | No buffer: 0 | Xon: 0 Xoff: 0 | Pause: 0 | Jumbo: 218k Noproto: 0 | Error symbol: 0 | Illegal symbol: 0 Bus overruns: 0 | Queue drops: 0 | LRO segments: 2060m LRO bytes: 4909g | LRO6 segments: 0 | LRO6 bytes: 0 Bad UDP cksum: 0 | Bad UDP6 cksum: 0 | Bad TCP cksum: 0 Bad TCP6 cksum: 0 | Mcast v6 solicit: 0 | Lagg errors: 0 Lacp errors: 0 | Lacp PDU errors: 0
cluster::> system node run -node "nodename" -command ifstat -a -- interface e4a (2 days, 10 hours, 22 minutes, 35 seconds) -- RECEIVE Total frames: 5161m | Frames/second: 24558 | Total bytes: 5152g Bytes/second: 24516k | Total errors: 35927 | Errors/minute: 10 Total discards: 0 | Discards/minute: 0 | Multi/broadcast: 26826k Non-primary u/c: 0 | - CRC errors indicate a receive side data issue
- CRCs result in packet loss due to re-transmissions and out-of-order packets received.