Fragmentation of TEP or RTEP traffic leads to packet drops in VMware NSX

Products

• VMware NSX

Issue/Introduction

• TEP or RTEP tunnels appear down or show intermittent "Degraded" status.
• Packet captures on receiving Transport Nodes show Geneve packets are fragmented.
  
  
• Traffic traversing the public Internet or intermediate routers is silently dropped.
  
  
• Packet reassembly error counters (e.g., rx_frag_error) increase on Edge or Host interfaces.

Environment

• VMware NSX 4.x, 3.x
  
  
• NSX Federation
  
  
• Multi-Site deployments communicating over public Internet or WAN

Cause

• NSX Transport Nodes (ESXi and Edges) send TEP and RTEP traffic with the "Don't Fragment" (DF) bit set.
  
  
• These nodes are not designed to reassemble fragmented encapsulated traffic.
  
  
• If intermediate network components clear the DF bit and fragment the packets, the receiving node drops the fragments because it cannot process a partial Geneve packet.

Resolution

1. Ensure all physical network components between sites maintain a consistent MTU that accounts for Geneve overhead (50 bytes).
   
   
2. Configure intermediate routers, firewalls, or VPN concentrators to respect the DF bit.
   
   
3. Align the NSX Global MTU setting with the minimum MTU supported by the physical path.
   
   
4. Verify connectivity using vmkping -d -s <size> <destination_IP> to confirm packets traverse without fragmentation.

If traversing the public Internet via RTEP, set the RTEP MTU to a value that allows for additional headers (e.g., IPsec) without exceeding the path MTU.

Additional Information

• Subscribe to this article to receive updates: https://knowledge.broadcom.com/external/article/275360.
  
  
• Review Guidance to Set Maximum Transmission Unit for calculation details.
  
  
• To speak to a Technical Support Engineer, see KB 206567 - Contact Broadcom support