Default gateway unreachable for VMs using L2VPN tunnels between NSX data centers

Products

• VMware NSX
• VMware NSX-T Data Center

Issue/Introduction

• L2VPN service has been created between two NSX environments within two separate software-defined data centers.
• Both sites stretch overlay (VNI) segments.
• DR downlink on either server or client side is configured as default gateway.
• While the VM's on the Client side of the tunnel will likely be able to communicate with VM's on the Server side and vice versa.
• VMs on one site can't ping the default gateway on the other site.

Environment

VMware NSX-T Datacenter

VMware NSX

Cause

• This issue occurs only when both the server and client are managed by NSX.
• VMs on one site cannot ping the default gateway on the other site. This is expected behavior of the ESXi host, because the destination mac-address of ICMP request packet is the mac-address of the VDR port, which is also locally present on the ESXI host as T1-DR instance, so traffic never leaves the ESXI host. This happens because the MAC addresses for NSX virtual distributed router are the same across all installations.

Resolution

Change the VDR MAC on either site so that both sites don’t use the same VDR MAC to fix this issue. 

Follow documentation to set the VDR MAC address correctly for both deployments.

NSX 3.1 - Change the MAC Address of NSX-T Virtual Distributed Router

NSX 4.1 - Change the MAC Address of NSX Virtual Distributed Router

 

Additional Information

Note: While changing the MAC address of NSX Virtual Distributed Router, if a VM is attached to an NSX overlay segment, you might observe a short disruption in the data plane. Kindly plan the change accordingly.