Multiple tunnel endpoints remain down on Esxi hosts due to vlan misconfiguration

Products

• VMware NSX

Issue/Introduction

Below symptoms may be observed on the ESXi cluster with multiple tunnel endpoints

• NIC teaming configured in uplinks with two or more network adapters on the ESXi host. 
• TEP tunnel (BFD) was getting established with one of the NIC adapter but not getting established with other NIC adapter. 
• When both the NIC adapters are enabled UP, it affects traffic on both NIC adapters and BFD status remain in INIT state on the ESXi

Environment

3.2.2.0.0.20737185

Cause

The cause of this issue could be physical networking issue like the uplinks on ESXi host are not connected to correct uplink switches or vlan not configured or mis-configuration on uplink switches.

Resolution

Follow below steps to troubleshoot the issue 

• Check BFD session status on the ESXi host by executing command "nsxdp-cli bfd session list". If BFD sessions are in "down or init" state proceed further troubleshooting 
• Shutdown one of the uplink by executing command "esxcli network nic down -n <vmnicX>"
• Check the assignment of uplinks on vmk10, vmk11 interfaces by executing command "nsxdp-cli vswitch instance list"
• Check if TEP IP on other ESXi host is reachable by executing command "vmkping -I vmkX -S vxlan -d -s 1572 <TEP IP of other ESXi host>" (vmkX is vmk10 or vmk11) 
• If the TEP IP is reachable then TEP tunnel will get established with the vmnicX. Also check BFD sessions established by executing command "nsxdp-cli bfd session list"
• Repeat above steps by shutting down the working NIC adapter and bringing UP the other NIC adapter, to bring up NIC execute command "esxcli network nic up -n <vmnicX>"

If the TEP IP is not reachable from either of the NIC adapter, then it could be an physical networking issue like not connected to correct port on the switch or vlan misconfiguration on uplink switches. Check the configuration on uplink switches for the ports that are connected with ESXi hosts.

Additional Information

For more information refer Troubleshooting NSX TEP/BFD Tunnels between ESXi hosts and Edges