BGP peering fails on NSX Edge nodes after vMotion

Products

• VMware NSX

Issue/Introduction

• NSX Edge nodes lose BGP peering with physical Top of Rack (ToR) switches.
• BGP status remains in 'Connect' or 'Active' state.
• Edge uplinks show ARP requests being sent but no ARP replies are received from the physical network.
• Physical switches cannot ping the Edge Node uplink IP addresses.
• Issue occurs immediately following a vMotion of the Edge VM to a different ESXi host.

Environment

VMware NSX

Cause

Edge VM vMotioned to a host where the physical network (ToR switches) does not have the required BGP VLANs configured or permitted on the new host's physical trunk ports. This results in a total loss of Layer 2 reachability for the Edge uplinks.

Resolution

1. Identify the ESXi host where the Edge VM was running prior to the connectivity loss.
2. vMotion the Edge VM back to the original host to restore connectivity.
3. To permanently resolve and allow vMotion, verify the physical switch port configurations for all hosts in the cluster.
4. Ensure BGP peering VLANs are tagged and permitted on all physical switch ports connected to the ESXi hosts.
5. Confirm Layer 2 reachability by pinging the physical peer IP from the Edge T0 Service Router (SR) VRF.
   Identify the VRF of the T0 SR, change into it and then perform the ping:
   > get gateways
Gateway UUID                           VRF    Gateway-ID   Name                              Type                        Ports   Neighbors
14######-####-####-####-###########1   2      1030         SR-T0-Global                      SERVICE_ROUTER_TIER0        7       5/50000
> vrf 2
> ping <TOR peer IP>