How does Gigabit Ethernet flow control work?
Applies to
- ONTAP 9
Answer
How Gigabit Ethernet Flow Control Works
An interface that is receiving data packets and is almost out of receive buffers will send a pause on
(Xoff) frame to its link partner. The link partner will stop transmitting data packets until it receives a pause off
(Xon) frame or a timeout expires. Enabling flow control reduces the possibility that one interface will overrun the other interface. A pause frame can impact all hosts connected to that interface. Upper layer protocols such as TCP should be preferably used instead of flow control, to control the flow of data.
Pause frames are only exchanged between two directly-connected link partners; that is, between a switch and an end station, or between two end-stations when they are connected back-to-back. Pause frames are NOT exchanged between two end-stations connected through a switch.
Some devices have one configuration option that enables the device to both send and receive flow control frames. Other devices have separate options for send and receive. When send
(or transmit
) is enabled, it means that the device will send pause frames. When receive
is enabled, it means that the device will respond to received pause frames.
Although the recommendation is to use flow control none (disabled), some benefits can still be realized by enabling send flow control on end-devices connecting into the network. Modern switch hardware is typically able to operate at higher speeds than standard NICs, and have additional queuing and buffering available. By setting the switch to receive on
and send off
, and the client to receive off
and send on
, the end-device is able to pause traffic flow and create backpressure on the switch.
Flow control will not eliminate problems when a Gigabit interface sends to a 100-Mbit client and the transfer size is very large. The difference in line speeds and a large transfer size cause too many packets to be queued on the switch's outbound 100-Mbit port, and the switch drops packets. The switch cannot send pause frames to the Gigabit interface, as that interface may also be sending to other interfaces and this would negatively affect performance on them.
Additional Information
TR-3802: Ethernet Storage Best Practices