Virtual Machines on NSX Overlay Segments are not able to communicate with external endpoints (i.e. IPs outside of NSX)

Products

• VMware NSX

Issue/Introduction

• Traceflow shows VM traffic dropping at the VDR port of the T1 gateway: Dropped due to ARP failure
  
• Note in the Traceflow output the Physical Hop Count starts at 0, and the last captured point is hop 0. This means the packet stopped on the same host it started.
  
  
  
• In the NSX GUI at System → Fabric -→Hosts, affected hosts show multiple or all TEP tunnels down and have a status of Degraded:
  
  
  
  
• At the ESX host command line, multiple tunnels show a down status, and all of the tunnel endpoints (TEPs) that are down belong to NSX Edges.
• To see the list of TEP endpoints at the ESX host CLI:
  nsxdp-cli bfd sessions list
• Note the local_state is DOWN only on REMOTE IPs that belong to the NSX Edge TEPs.

Environment

VMware NSX

Cause

• The issue occurs due to Layer 3 inter-VLAN routing failure in the physical network infrastructure between Host TEP and Edge TEP networks.
• When Host and Edge TEPs are configured on separate VLANs (recommended practice), proper inter-VLAN routing must be established in the physical network. 

Resolution

Follow the recommended resolution steps in TEP tunnels are down between ESXi Transport Node and Edge Transport Node. (Alarm: Event type: Faulty TEP).