Tunnels between NSX-T Edge and ESXi nodes displays as Down even when there are no connectivity issues

Products

• VMware NSX

Issue/Introduction

• Edge virtual machines reside in the ESXi host prepared for NSX-T as well as compute workloads being on the same host switch
• Tunnels between NSX-T Edge and ESXi nodes displays as Down even when there are no connectivity issues.
  
  

Environment

VMware NSX-T Data Center 3.x

VMware NSX 4.x

Cause

This issue occurs because the TEPs of the ESXi compute transport nodes and Edge VMs are in the same subnet.

For tunnels between an Edge VM and its outer ESX compute transport nodes, BFD packets sent from the ESXi node will be delivered to the Edge VM without encapsulation, and the Edge VM drops the BFD packets without encapsulation.

For tunnels between an Edge VM and an ESX node which is not the host of certain Edge VM, BFD packets sent from the ESXi node is dropped by the ESXi node hosting the Edge VM because these packets are not destined to the local TEPs of the ESXi node hosting the Edge VM.

Resolution

 

This issue is resolved in VMware NSX 4.2, available at Broadcom downloads.

Workaround:
To work around this issue, ensure to place the Edge TEPs on a different subnet/VLAN than the TEPs in the ESXi compute transport nodes.

Note: If bridging is enabled, then the workaround likely will not work, though to be sure that the TEPs work correctly on later versions(4.2 and above), the TEPs still need to be on a different subnet/VLAN.
The bridge should be on it's own subnet, otherwise announcements for the TEP can go across the bridge making it appear as though there are duplicate mac addresses, which is what causes the TEP's to go down.

 

Additional Information

For 4.2 and later if the TEPs are on their own VLAN, see page 325 of the design guide (page 325).