NSX Edge RTEP Tunnel Fails Due to Physical Underlay Source NAT Modifying Packets

Products

• VMware NSX

Issue/Introduction

An anomaly is observed in RTEP communication between NSX Edges where the source IP address of an ARP request is altered in transit. Specifically, an ARP request originating with source IP 172.30.52.7 is received by 172.30.53.6 with a modified source IP of 172.30.52.1. The destination IP remains  172.30.53.6.

This behavior is unidirectional; reverse path ARP requests from the destination back to the source are received without modification. Simultaneous packet captures on both edge nodes demonstrate the mutation:

• Outbound capture on Source Edge shows the ARP request generated correctly with its assigned RTEP IP  172.30.52.7.

• Inbound capture on Destination Edge shows the received ARP request with the Source IP 172.30.52.1 (changed to the subnet's default gateway IP.)

Environment

VMware NSX

Cause

A physical underlay network device is erroneously applying Source NAT (SNAT) to the RTEP traffic exiting the Edge subnet, altering the source IP address in transit.

Resolution

NSX RTEP overlay tunnels require direct, un-NATted line-of-sight between Edge nodes. To resolve this issue:

1. Engage the physical network infrastructure team to trace routing and security policies on the physical devices bridging the affected <REDACTED_IPS> subnets.

2. Identify the specific physical gateway router that is modifying the packets (typically the device holding the modified source IP).

3. Remove or bypass the NAT/Proxy ARP configuration for the RTEP subnets to ensure end-to-end un-NATted communication.

4. Verify bidirectional ARP resolution and successful tunnel initialization via NSX Edge packet captures.

Additional Information

For more information on Edge node underlay network requirements, refer to the VMware NSX Reference Design Guide - NSX Edge Networking Setup

For VTEP scenario kindly refer this KB article --NSX Edge VTEP Tunnel Failure: Unintended Source NAT on Physical Underlay Modifying Transit Packets