NSX Edge VM network connectivity loss during physical switch maintenance

Products

• VMware NSX

Issue/Introduction

• North-South network connectivity for virtual machines on NSX overlay segments drops completely when a redundant physical Top-of-Rack (ToR) switch reboots or is powered off.
• ESXi host management and vMotion connectivity remain active, but Edge VM traffic is lost.
• NSX Edge VM datapath interfaces (eg: fp-eth0, fp-eth1) show in a fallback state when running the nsxdp-cli vswitch instance list command on the ESXi host.
• NSX Manager UI displays BFD alarms: Control Detection Time Expired.

Environment

VMware NSX

VMware Cloud Foundation (VCF)

Cause

This is expected behavior given the applied vSphere Distributed Switch (vDS) Teaming and Failover configuration.

When the Distributed Port Groups (DPGs) supporting the NSX Edge VM datapath interfaces (fp-eth0 and fp-eth1) have the secondary physical uplink placed in the Unused section rather than Active or Standby, the hypervisor is explicitly prohibited from failing over traffic to that uplink.

When the primary physical link fails during maintenance or an outage, the Unused policy prevents the ESXi host from failing over the Edge VM's virtual NIC (vNIC) traffic to the surviving physical uplink. This severs the Geneve encapsulation path and BFD sessions for all North-South traffic handled by that Edge node.

Resolution

Note: This procedure applies specifically when Edge VM datapath interfaces are hosted on standard vSphere Distributed Switch (vDS) Distributed Port Groups. If your Edge VMs are attached to NSX VLAN Segments, see the Additional Information section below.

To restore redundant path failover for Edge VMs on vDS Distributed Port Groups:

1. Log in to the vSphere Client.

2. Navigate to Inventory > Networking.

3. Select the Distributed Port Group(s) assigned to the NSX Edge VM datapath interfaces.

4. Right-click the port group and select Edit Settings.

5. Navigate to the Teaming and Failover tab.

6. Move the physical uplink currently in Unused uplinks into the Standby uplinks (or Active uplinks, depending on your teaming requirements).

7. Click OK.

8. Repeat these steps for all port groups backing Edge VM datapath interfaces.

Once applied, ESXi will instantly shift the Edge VM's vNIC traffic to the surviving physical uplink during a ToR failure, maintaining Geneve tunnels and North-South traffic flow.

Additional Information

If the NSX Edge VM datapath interfaces are attached to NSX VLAN Segments (N-VDS / vDS-backed segments) rather than vSphere DPGs, the failover policy is controlled within NSX rather than vCenter:

• NSX Segment Fix: Adjust the Uplink Profile assigned to the Edge Transport Nodes. Ensure both physical uplinks are defined under Active/Standby lists or utilize Named Teaming Policies to explicitly map traffic paths across the redundant physical uplinks.

• Architectural Reference: For comprehensive VCF network design specifications regarding Edge VM deployment and uplink mapping, review the Broadcom VCF Design Guidance.