BFD tunnels down between Edge nodes in Collapsed Cluster

Products

• VMware NSX

Issue/Introduction

All BFD tunnels with the Edge nodes are down, however ESXi tunnels may remain unaffected.

The Edge nodes are in a Collapsed Cluster (all Edge and Manager nodes are deployed on hosts prepared for NSX).

The Edge nodes are using an Uplink Profile on the TEP interface configured for a VLAN tag that is also configured on the segment used by the node's TEP interface.

• To see the VLAN configured for the Uplink Profile on the Edge:
  1. In the NSX UI, navigate to System -> Nodes and "Edit" the node in question
  2. Note the Uplink Profile for the TEP interface's switch
  3. Cancel out of the editing menu without making any changes
  4. Navigate to System -> Profiles and note the VLAN configured for the Edge's Uplink Profile recorded in step 2 above
• To see the VLAN configuration for the Edge's TEP interface:
  1. In the NSX UI, navigate to Networking -> Connectivity -> Segments and expand the segment used by the Edge's TEP interface
  2. Note the VLAN

Environment

VMware NSX 4.X

Cause

A misconfiguration between the Edge's TEP interface and the segment it utilizes leads to the virtual switch to drop all traffic to/from the TEP interface.

This loss of all packets, including ARP, before it enters the physical network in turn causes all Layer-2 connectivity to fail.

Resolution

Change the segment used by the Edge TEP interfaces to be trunked for all VLANs to allow for traffic that is already tagged to be passed normally.

1. In the NSX Manager UI, navigate to Networking -> Connectivity -> Segments
2. Select the vertical ellipses next to the TEP interface segment and click "Edit"
3. Remove the existing VLAN and add a new VLAN of "0-4094" which will trunk the segment for all VLANs
4. Click "Save" and then "Close Editing"

NOTE: The tunnel status may take a few minutes to re-establish connectivity, but the status can be monitored in System -> Fabric -> Nodes and clicking "Refresh" on the bottom-right of the screen.

Additional Information

This configuration is essentially Virtual Guest Tagging, which is described in the following KB: VLAN configuration on virtual switches, physical switches, and virtual machines

See also Troubleshooting NSX TEP/BFD Tunnels between ESXi hosts and Edges for further information and possible issues.