Configuring jumbo frame support on NSX for vSphere and VCNS

Products

• VMware NSX

Issue/Introduction

This article provides information about changing Maximum Transmission Unit (MTU) settings to support jumbo frames with logical switches on VMware NSX for vSphere 6.x and VMware vCloud Networking and Security (VCNS) 5.x.

Symptoms:

Environment

VMware NSX for vSphere 6.1.x
VMware vCloud Networking and Security 5.5.x
VMware vCloud Networking and Security 5.1.x
VMware NSX for vSphere 6.0.x
VMware NSX for vSphere 6.2.x

Resolution

By default, the source and destination virtual machine MTU is 1500 bytes. For NSX prepared cluster with VXLAN, the default vSwitch MTU is adjusted to 1600 bytes to accommodate larget VXLAN packet size.

The recommended MTU for each NSX vDS switch is 1600 or higher.

 

Unless an alternative value is configured, a default MTU size of 1600 bytes is assigned to VTEP vmknics.

 

Notes:

• Ensure the correct value is configured on the VTEP vmknic as the NSX/VCNS distributed logical router does not fragment or reassemble frames.
  Headers needed:
  Content-Type : application/xml
  Accept: application/xml
  Basic Auth

 

1. GET on https://<NSX/VCNS-mgr-IP>/api/2.0/vdn/switches
   
   Note:
   • This lists all virtual distributed switches in the system used for VXLAN.
   • The objectID of the VDS with the MTU to be changed. The objectID will be similar to <objectId>dvs-100</objectId>.
     
     
2. PUT on https://<NSX/VCNS-mgr-IP>api/2.0/vdn/switches
   
   BODY:
   <vdsContext>
   <switch>
   <objectId>dvs-100</objectId>
   </switch>
   <mtu>9000</mtu>
   </vdsContext>
   
   Note: Replace dvs-100 with the objectId found in step 1.
   
   When this call succeeds, a 200 OK status is returned.

To update the MTU on existing VTEPs or on the distributed virtual switches, use the vCenter Server UI. Any changes are not expected to affect running VMs with the following exceptions:

• Increasing the DVS/VTEP MTU beyond the MTU of the physical network can cause traffic disruption.
• Decreasing the MTU below (MTU of VM + Encapsulation Header Size) can cause ESX to drop packets, as we do not fragment packets if there is not enough room to add the encapsulation headers. For VXLAN over IPv4, the encapsulation headers are 50 bytes.​

For more information about the logical switch configuration, see the Logical Switch Scenario section in the NSX Administration Guide.

Additional Information

For more information, see VMware VXLAN Deployment Guide.