Skip to main content
NetApp Knowledge Base

High latency from cluster interconnect with port errors

Views:
132
Visibility:
Public
Votes:
0
Category:
ontap-9
Specialty:
perf
Last Updated:

Applies to

ONTAP 9

Issue

cluster1::>  qos statistics volume latency show -volume vserver_vol01 -vserver  vserver1
Workload            ID    Latency    Network    Cluster       Data       Disk    QoS Max    QoS Min      NVRAM      Cloud  FlexCache    SM Sync         VA
--------------- ------ ---------- ---------- ---------- ---------- ---------- ---------- ---------- ---------- ---------- ---------- ---------- ----------
-total-              -    10.71ms   520.00us     9.63ms   465.00us    88.00us        0ms        0ms     3.00us        0ms        0ms        0ms        0ms
vserver_vol01..   7948  1247.00us    48.00us  1140.00us    25.00us    33.00us        0ms        0ms     1.00us        0ms        0ms        0ms        0ms
-total-              -    26.10ms     5.74ms    19.53ms   753.00us    71.00us        0ms        0ms     6.00us        0ms        0ms        0ms        0ms
vserver_vol01..   7948   479.00us    41.00us   259.00us    26.00us   150.00us        0ms        0ms     3.00us        0ms        0ms        0ms        0ms
-total-              -   125.97ms   823.00us   124.86ms   187.00us    86.00us        0ms        0ms     8.00us        0ms        0ms        0ms        0ms
vserver_vol01..   7948  1466.00us    62.00us  1238.00us    45.00us   109.00us        0ms        0ms    12.00us        0ms        0ms        0ms        0ms

  • CRC errors reported on EMS logs :

Mon Dec 12 13:31:37 +0530 [cluster1: vifmgr: vifmgr.cluscheck.hwerrors:alert]: Port e0e on node cluster1-01 is reporting a high number (at least 1 per 1000 packets) of observed hardware errors (CRC, length, alignment, dropped).
Mon Dec 12 14:31:37 +0530 [cluster1: vifmgr: vifmgr.cluscheck.hwerrors:alert]: Port e0e on node cluster1-01 is reporting a high number (at least 1 per 1000 packets) of observed hardware errors (CRC, length, alignment, dropped).
Mon Dec 12 15:31:38 +0530 [cluster1: vifmgr: vifmgr.cluscheck.hwerrors:alert]: Port e0e on node cluster1-01 is reporting a high number (at least 1 per 1000 packets) of observed hardware errors (CRC, length, alignment, dropped).

  • Error symbol counts in ifstat -a output of the cluster interconnect port (e0e) are increasing:

 -- interface  e0e  (54 days, 3 hours, 15 minutes, 51 seconds) --

RECEIVE
Total frames:    11316m | Frames/second:    2420  | Total bytes:     11802g
Bytes/second:     2523k | Total errors:      290m | Errors/minute:    3732
Total discards:      0  | Discards/minute:     0  | Multi/broadcast:  6794k
Non-primary u/c:     0  | Errored frames:      0  | Unsupported Op:      0
CRC errors:        273m | Runt frames:         0  | Fragment:          797k
Long frames:        36  | Jabber:              4  | Length errors:       0
Alignment errors:    0  | No buffer:           0  | Pause:               0
Jumbo:             105m | Error symbol:    16247k | Bus overruns:        0
Queue drops:         0  | LRO segments:     4105m | LRO bytes:       11181g
LRO6 segments:       0  | LRO6 bytes:          0  | Bad UDP cksum:       0
Bad UDP6 cksum:      0  | Bad TCP cksum:       0  | Bad TCP6 cksum:      0
Mcast v6 solicit:    0  | Lagg errors:         0  | Lacp errors:         0
Lacp PDU errors:     0

Sign in to view the entire content of this KB article.

New to NetApp?

Learn more about our award-winning Support

NetApp provides no representations or warranties regarding the accuracy or reliability or serviceability of any information or recommendations provided in this publication or with respect to any results that may be obtained by the use of the information or observance of any recommendations provided herein. The information in this document is distributed AS IS and the use of this information or the implementation of any recommendations or techniques herein is a customer's responsibility and depends on the customer's ability to evaluate and integrate them into the customer's operational environment. This document and the information contained herein may be used solely in connection with the NetApp products discussed in this document.
  • Was this article helpful?