What are the flow control best practices for Ethernet?

Applies to

ONTAP 9

Answer

• Follow flow control best practices when experiencing excessive pause frames, xon / xoff frames, or packets being re-queued, as seen in ifstat -av.

- interface  e1a  (0 hours, 0 minutes, 40 seconds) --
RECEIVE
 Frames/second:    1568  | Bytes/second:    27607k | Errors/minute:       0
 Discards/minute:     0  | Total frames:      248k | Total bytes:      4222m
 Total errors:        0  | Total discards:      0  | Multi/broadcast:     2
 No buffers:          0  | Non-primary u/c:     0  | LRO segments:      172k
 LRO bytes:        3929m | L2 terminate:        0  | Tag drop:            0
 Vlan tag drop:       0  | Vlan untag drop:     0  | Vlan forwards:       0
 Vlan broadcasts:     0  | Vlan unicasts:       0  | CRC errors:          0
 Runt frames:         0  | Fragment:            0  | Long frames:         0
 Jabber:              0  | Bus overruns:        0  | Queue drop:          0
 Xon:                 25  | Xoff:                25  | Jumbo:           68353
 No buf hi:           0
TRANSMIT
 Frames/second:    1055  | Bytes/second:      154k | Errors/minute:       0
 Discards/minute:     0  | Total frames:      166k | Total bytes:     44356k
 Total errors:        0  | Total discards:      0  | Multi/broadcast:     2
 Queue overflows:     0  | No buffers:          0  | Frames queued:       0
 Buffer coalesces:  854  | MTUs too big:        0  | TSO segments:        0
 TSO bytes:           0  | TSO6 segments:       0  | TSO6 bytes:          0
 LTM fastp calls:   166k | LTM busy calls:      0  | LTM update calls:  160
 LTM yield calls:     0  | LTM drain rss:       0  | LTM total calls:   166k
 Xon:                 0  | Xoff:                0  | Jumbo:            3765
 TSO non-TCP drop:    0  | Split hdr drop:      0  | Pktlen:              0
 Timeout:             0  | Timeout1:            0
LINK_INFO
 Current state:       up | Up to downs:         6  | Speed:           10000m
 Duplex:            full | Flowcontrol:       full
RSS
 Itable updates:      0  | Hash errors:         0
Queue 0
 Tx frames:       11780  | Tx bytes:        22306k | Rx frames:          59
 Rx bytes:          103k | Requeued:            0  | Hi pri intr:     10735
Queue 1
 Tx frames:        1073  | Tx bytes:          143k | Rx frames:        4484
 Rx bytes:        33895k | Requeued:         3165  | Hi pri intr:      4160
Queue 2
 Tx frames:         306  | Tx bytes:        33404  | Rx frames:        3469
 Rx bytes:         3539k | Requeued:         3007  | Hi pri intr:      3831
Queue 3
 Tx frames:         153k | Tx bytes:        21872k | Rx frames:         240k
 Rx bytes:         4184m | Requeued:          801  | Hi pri intr:       143k

• It is recommended to match the flow control setting throughout each network path, including intercluster ports, data switches, and data ports. (excluding intracluster /cluster network ports)
• A subsequent effort by Cisco, priority flow control (defined in IEEE 802.1Qbb), was intended to replace802.3x PAUSE and provide a mechanism that could be controlled independently for each class of service defined by IEEE802.1p.  Instead of pausing all traffic on a network link, priority flow control allows the administrator to selectively pause traffic according to its class of service.
• Historically (7-Mode/clustered Data ONTAP), NetApp had recommended that flow control be disabled on all network ports within an ONTAP cluster. This approach is no longer the case. Guidance in this area has since changed, the new recommended best practice is as follows:
  • Disable flow control on cluster network ports in the ONTAP cluster.
  • Flowcontrol on the remaining network ports (the ports that provide data, management, and intercluster connectivity) should be configured to match the settings within the rest of your environment.

Additional Information

N/A