There seems to be a growing argument lately over network optimization and the use of Jumbo Frames. With the cost of Gigabit networking continuing to go down, and 10 Gigabit (and greater) networks becoming more affordable, the impact of this decision is becoming more and more important.
Knowing how and when to use jumbo Ethernet frames is key to understanding how to optimize network performance.
As Ethernet networks grow along with increasing bandwidth requirements, so does the requirement for a more optimal network.
Gigabit networks which were at one point part of the core infrastructure, have given way to Ether-Channel and larger 10 Gigabit, 40 Gigabit and now even 100 Gigabit networks.
All of these bigger networks allow for more data to be transferred in the same amount of time.
Most people will incorrectly state that the network speed is increasing, but in reality it is the amount of data transferred not the speed. All networks run at about the same speed bound by Einstein’s Relativity theory and the speed of light.
One thing that isn’t changing much is the Ethernet frame size. Many inexperienced network engineers will setup a network switch right out of the box and never adjust the MTU size to accommodate for the type of Ethernet network they will run.
Standard Ethernet Frames
Most gigabit Ethernet networks run fine straight out of the box using standard Ethernet frames. The standard Ethernet frame has a size of 1542 bytes with the maximum usable payload on a standard Ethernet frame of 1500 bytes.
This works great on most gigabit networks but in some circumstances (depending on the application) increasing the frame size to 9000 bytes (Jumbo Frames) can improve the overall performance.
What are Jumbo Frames and When Should You Use Them
Jumbo frames are any Ethernet frame that has a payload larger than 1500 bytes. When you think Jumbo frames most people think of a frame that can carry a maximum payload of 9000 bytes.
Most Ethernet switches today support jumbo frames, but must be configured to do so in order for this to work correctly.
Why Do We Need Jumbo Frames
Jumbo frames were introduced to help improve the overall performance of Ethernet networks using gigabit and faster Ethernet networks. Each frame that a switch has to process uses resources on the switch. Enabling jumbo frames, increases the frame size, reducing the number of frames the switch needs to process and reduces the overhead and CPU cycles needed on the switch.
Knowing Your Applications
Understanding the applications that ride on your network will help you understand if enabling jumbo frames can help your overall network performance.
Applications such as NFS that operate using 8KB datagrams are perfect for jumbo frames. Storage networks are also a perfect example of this. 8KB datagrams fit perfectly into the 9000 byte payload of a jumbo frame.
Problems with Jumbo Frames
All is not roses in the land of Jumbo Frames. Jumbo frames aren’t for all situations. Low bandwidth circuits or applications that are highly sensitive to latency, jitter, or delay (such as multimedia applications, VoIP, etc.,) will not work well with jumbo frames.
How to Enable Jumbo Frames on a Cisco Switch
You can enable jumbo frames on an interface by interface basis or you can enable jumbo frames on the entire switch.
Enabling jumbo frames on a Cisco switch is fairly simple. Depending on which device you have will depend on the exact commands you need to use to enable jumbo frames.
On Cat 6500 Series Switches
6509 (config) # int gigabitEthernet 1/1
6509 (config-if)# mtu ?
MTU size in bytes
6509(config-if)# mtu 9216
6509# show interfaces gigabitEthernet 1/1
GigabitEthernet1/1 is up, line protocol is up (connected)
Hardware is C6k 1000Mb 802.3, address is 0007.0d0e.640a (bia 0007.0d0e.640a)
MTU 9216 bytes, BW 1000000 Kbit, DLY 10 usec,
reliability 255/255, txload 1/255, rxload 1/255
Understanding your applications is key to knowing if enabling Jumbo Frames will improve your performance. In most situations and for most modern Gigabit networks, enabling Jumbo Frames will help reduce the load on your switches infrastructure.
Remember the 9000 byte MTU is the MAXIMUM transmission unit size. Ethernet is flexible layer 2 protocol and will adjust for smaller frame sizes. However latency sensitive applications must be treated separately and will not function optimally on jumbo frame networks. Segmenting standard Ethernet frame networks to their own VLAN is the best way to ensure optimal network performance for all applications and is the best way to allow for both types of networks to coexist.
Have comments or question? Leave a comment below and let’s talk about jumbo frames.
A very good article. Just wanted to share one point of view:
I believe based on the technology and the support for jumbo frames by switches in different manner the statement “Jumbo Frames must be configured correctly end-to-end. This means on the vSwitch, vmkernel interfaces, physical switches, endpoint – everything must be configured to support the larger frames end-to-end” is not accurate.
There are cases from a number of vendor deployments where physical switches can be segregated into switching domains may be like 6880 and Nexus series so the term ‘end-to-end’ will confuse even the mature engineers and architects.
There are switch from so many vendors that was not exactly the case 5 years ago; that can even have different MTU for different port-channel / LACP groups.
Imagine a storage connected with a switch over a port channel and carry ISCSI Vlan that leads to a VMKernal on a vDS. If someone does not understand all component in this path having a larger MTU and confuses it with the need to have a large MTU for the whole switch; they might be in for a blocker when it comes to impement Jumbo frames.
Question on jumbo frame?
I have an host configured with MTU=9000 bytes and switch with 1500 bytes. Hence when packet traverse from host through switch. Do any packet loss will occur. please explain . If so tell me how to track that using wireshark tool.
I have two Cisco switches, one I have all my IP phones and printers, the other is all my computers and servers connected with Gigabit Ethernet, can I enable jumbo frames on the the second and leave the other at 1500 or will that still create problems?
Sometimes there are more than one VLAN on an interface, where one VLAN may not be suitable for jumbo frames but others are. Would it be best practice in this scenario to stick with a standard MTU size of 1500 on switch trunk interfaces?
Again, I believe it depends on the applications. In the case of Voice trunks and data trunks co-existing on the same interface, yeah I would opt for 1500 byte frames.