What is Buffer-to-Buffer credit management?

What are Buffer-to-Buffer Credits (BB_Credits) and how are they used?

• Fiber Channel uses the BB_Credit mechanism for hardware-based flow control. This means that a port has the ability to pace the frame flow into its processing buffers.
• Buffer-to-buffer credit is used to manage flow-control and determine how many frames can be sent to the recipient switch device without saturation. This mechanism eliminates the need of switching hardware to discard frames due to high congestion.
• The number of credits a port has is equal to the number of frames a port can transmit before obtaining an acknowledgement that the frame was received. BB_Credit management occurs between any two Fiber Channel ports that are connected.

Example:

• One N_Port and one F_Port
• Two E_Ports
• Two N_Ports in a point-to-point topology
• In Arbitrated Loop different modes

The standard provides a frame-acknowledgement mechanism in which an R_RDY (Receiver Ready) primitive is sent from the receiving port to the transmitting port for every available buffer on the receiving side.

The transmitting port maintains a count of free receiver buffers, and will continue to send frames if the count is greater than zero.

BB_Credits are used by Class 2 and Class 3 services and rely on the receiver sending receiver-readies (R_RDY) to the transmitter. Node pairs communicate their number of credits available during FLOGI/PLOGI. This value is used by the transmitter to track the consumption of receive buffers and pace transmissions if necessary. The FC switches track the available BB_Credits in the following manner:

• Before any data frames are sent, the transmitter sets a counter equal to the BB_Credit value communicated by its receiver during FLOGI
• For each data frame sent by the transmitter, the counter is decremented by one
• Upon receipt of a data frame, the receiver sends a status frame (R_RDY) to the transmitter indicating that the data frame was received and the buffer is ready to receive another data frame
• For each R_RDY received by the transmitter, the counter is incremented by one 

Note:VC_RDY is a Brocade proprietary version of R_RDY used with Virtual Channels (VC) technology. In this document, the term 'R_RDY' is used to refer to either R_RDY or VC_RDY.

How many BB_Credits are required at a given link speed and length?

The calculations listed do not apply to MetroCluster FC backend configurations.

• During fabric login, N_Ports and F_Ports at both ends of a link establish its BB_Credit. Each port states the maximum BB_Credits that it can offer, and the lower of the two is used.
• The Host Bus Adapters (HBAs) on the host and the target have a fixed number of BB_Credits and cannot be modified. The Fiber Channel switch however calculates the BB_Credit for each port internally taking into account the speed and the link distance specified.
• The formula for calculating the number of BB_Credits necessary for a given distance (assuming maximum frame size) is:

(X * Y / 2) + 6 = B where,

B = number of BB_Credits required

X = distance in km* (to be safe, X is set as 1.5 * actual physical distance to be supported)

Y = speed

6 = the number of buffers required for fabric services per port. This number is always 6.

Note: If QoS is enabled on the links, 14 more credits will have to be taken into account which is required for QoS management.

When the maximum number of BB_Credits available for a given switch, as in the preceding table is known, the formula to arrive at the maximum theoretical distance relative to BB_Credits can be changed:

X = (B-6) / (Y / 2) where:

X = maximum theoretical distance*

B = BB_Credits available

Y = speed

For example, the maximum theoretical distance for Brocade 200E running 1 ISL at 4Gbps is determined as follows:

x = (144 – 6) / (4 / 2)

x = 138/3

x = 69 km 

The number of required buffers can also be calculated based on frame sizes. In cases where the frame size is average, for example 1024 bytes, allocate twice the BB_Credits or give twice the distance in the long-distance LS configuration mode.

1.Use the following formula to calculate the value for the desired_distance needed for the switch to determine the number of BB_Credits to allocate:

• desired_distance = roundup to nearest integer [(real_estimated_distance * 2112) / average_payload_size]
• Where average_payload_size = 1024 bytes
• This example uses 200 km for the real estimated distance.
• desired_distance = roundup to nearest integer [(200 * 2112) / 1024] = 412
• On Brocade switches, when configuring the LS mode with the portcfglongdistance command, enter a desired_distance value of 412 for an actual 200 km
• link connected to an 4 Gbps E_Port. This causes Fabric OS to allocate the correct number of BB credits.

2.Determine the speed that will be used for the long-distance connection. This example uses 8 Gbps.

3.Look up the data_rate value for the speed of the connection. A 1 Gb Fiber Channel link with 8b/10b encoding at a Baud rate of 1.0625Gb/s can deliver exactly 1 Gbps     of data flow. The Fiber Channel gigabit values are as below.

• 1.0625 for 1 Gbps
• 2.125 for 2 Gbps
• 4.25 for 4 Gbps
• 8.5 for 8 Gbps

4.Use the following formula to calculate the number of BB_Credits to allocate:

• BB_Credits = roundup [desired_distance * (data_rate / 2.125)]
• Using the values for desired_distance and data_rate from step 1 and step 3, the value for BB_Credits is calculated as follows:
• BB_Credits = roundup [(412 * 4.25) / 2.125] = 824

What happens if there are not enough BB_Credits available?

• Traffic can still flow in an FC network when insufficient buffers are available. They will just operate at slower line rates. This condition is known as BB_Credit starvation and will still allow traffic to flow as long as BB_Credits are not lost.
• In fact, if a BB_Credit pool reaches zero, the port can no longer send frames until the credits are replenished or until the link is reset.
• Fabric Segmentation might occur if BB_Credits do not match on each end of the link and that might cause the inter switch links to not come up. Check the error messages in the logs at both ends and determine why the ISL is failing to come up.

Note: If a given implementation has a high percentage of FC frames of sizes significantly lesser than the maximum size, then more BB_Credits might be required.

Buffer Credit Recovery

This feature is supported only on E-ports and allows switches to automatically recover BB_Credits that were accidentally 'lost' over time. BB_Credit recovery prevents link performance degradation that might otherwise occur due to loss of credits over time. For more information, see the switch vendor documentation.