What is the theoretical maximum throughput of a Gigabit Ethernet interface?
Applies to
- Clustered Data ONTAP 8
- Data ONTAP 7 and earlier
Answer
During the course of performance benchmarking, the issue of maximum theoretical throughput often rears its ugly head. What is the theoretical maximum for 1Gbps?
Note: The following are approximations that should be used as a guideline, and not as a scientific fact.
We know a couple of things about Ethernet that will allow us to calculate the theoretical maximum throughput. All frames must have a 8byte preamble, a 12byte inter-frame gap, and a minimum length of 64 bytes which includes Destination MAC (6 Bytes), Source MAC(6 Bytes), Protocol Type (2Bytes), Payload (46 Bytes) and CRC(4 Bytes). Given this, the frame size including the preamble and inter-frame gap is 84bytes (8+12+64).
The number of frames per second can be calculated as:
Rate / Frame size = frames per second
1000Mbps / (84bytes x 8) = frames/s
-OR-
1,000,000,000 bits / 672 bits = 1,488,000 frames/s.
As a result, the maximum theoretical throughput is calculated as:
frames per second x frame size
1,488,000 x 512 bits** = 761Mbps
Note: 64byte frame x 8 bits = 512bits
However, we also lose some bandwidth from the preamble and the inter-frame gap. They can be calculated as follows:
Preamble (recall that it is 8bytes):
frames per second x 8 bytes x 8 bits (to convert it to Mbps)
1,488,000 x 8 x 8 = 95Mbps
Inter-frame gap (recall that it is 12bytes):
frames per second x 12 bytes x 8 bits (to convert to Mbps)
1,488,000 x 12 x 8 = 143Mbps
So the actual maximum, given 64 byte frames, is 523Mbps (761 - 95 - 143) or 65MB/s.
Now, let's do the calculation quickly using a 1518byte frame.
Add in the preamble and the inter-frame gap:
8+12+1518= 1538
1000Mbps / (1538bytes x 8) = 81,274 frames/s
81,274frames/s x 12144bits*** = 986Mbps
Note: 1518bytes x 8 bits = 12,144bits
Preamble overhead: 81,274 x 8 x 8 = 5Mbps
Inter-frame gap overhead: 81,274 x 12 x 8 = 7Mbps
So the max throughput, given a 1518 byte frames, is 974Mbps (986 - 5 - 7) or 121MB/s.
Note that these numbers do not include Ethernet frame, IP, TCP or UDP overhead, so we will take an additional hit there.
Let's take a look at a 9k MTU (jumbo frames) with all accompanying overhead for TCP.
Frame size = 9000bytes
Inter-frame gap=12bytes
Ethernet Preamble=8bytes
Ethernet Header=14bytes
Ethernet FCS=4bytes
IP Header = 20bytes
TCP Header = 20bytes
TCP Options = 12 bytes
Frames per second:
9000+12+8 = 9020 x 8 = 72,160 bits
1,000,000,000 bits / 72,160 bits = 13,858 frames/s
Max throughput with no overhead:
13,858 x 72,000 = 997Mbps
Preamble overhead:
13,858 x 8 x 8 = .886
Inter-frame gap:
13,858 x 12 x 8 = 1.33Mbps
Ethernet Header overhead:
13,858 x 14 x 8 = 1.55Mbps
Ethernet first customer ship (FCS) overhead:
13,858 x 4 x 8 = .443Mbps
IP Header overhead:
13,858 x 20 x 8 = 2.21Mbps
TCP Header overhead:
13,858 x 20 x 8 = 2.21Mbps
TCP Options overhead:
13,858 x 12 x 8 = 1.33Mbps
Theoretical throughput of Gigabit Ethernet with jumbo frames, and using TCP:
997Mbps - .886 - 1.33 - 1.55 - .443 - 2.21 - 2.21 - 1.33 = 987Mbps or 123MB/s.
The approximate throughput for Gigabit Ethernet without jumbo frames and using TCP is around 928Mbps or 116MB/s. However, this is still not an accurate representation of what your customer can expect in the real world. Other factors will influence the throughput. These include, but are not limited to, file sizes, types of transactions, cache hits/misses, cpu utilization, network utilization, disk utilization, protocol (Network File System (NFS), Common Internet File System protocol (CIFS), etc), client type, kernel version, etc.
Additional Information
N/A