NSX overlay ping with DF bit set throws error: 36 bytes from IP: frag needed and DF set (MTU 1500)

Products

• VMware NSX

Issue/Introduction

• VM's on NSX overlay segments may experience packet drops when transmitting large payloads, especially when the "Don't Fragment" (DF) bit is set. This article outlines how to identify and resolve MTU bottlenecks, including critical global networking configuration mismatches.
  
  
• When testing connectivity, the following errors may appear, indicating that the packet size exceeds the allowed MTU. Note: The IP address observed in the output (typically in the 100.64.x.x range) represents an internal NSX Tunnel Endpoint (TEP) or Gateway interface:
  
  From 100.x.x.x icmp_seq=1 Frag needed and DF set (mtu = 1500)
ping: sendmsg: Message too long
ping: local error: Message too long, mtu=1500

Environment

VMware NSX

Cause

Geneve encapsulation adds approximately 50 bytes of overhead to the original frame. Connectivity failures occur when the total packet size exceeds the MTU of the underlying physical network or the Distributed Virtual Switch (DVS). A common root cause is a misconfiguration in the Global Networking settings, where the Global Gateway interface is left at the default 1500 MTU rather than being configured for Jumbo Frames (9000 MTU).

Resolution

1. Identify the MTU bottleneck by running a TEP-to-Gateway ping test from the ESXi host: vmkping ++netstack=vxlan -I vmkX -d -s 1572 [Gateway_IP]
2. Navigate to Networking > Global Networking Config.
3. Verify the MTU setting for the Global Gateway interface. If it is set to 1500, update it to Jumbo (9000) to match the overlay requirements.
4. Coordinate with your network team to increase the MTU on all physical switch ports and routed interfaces in the path to support Jumbo Frames.
5. Ensure the physical infrastructure supports a minimum MTU of 1600 bytes (9000 bytes is recommended as best practice).
6. Verify connectivity to the Edge Node TEP and the destination Guest VM.

Additional Information

VMware NSX MTU Guidance