VM experiences network interruption in an NSX environment

Products

• VMware vSphere ESXi
• VMware NSX

Issue/Introduction

• Virtual Machines (VMs) in an NSX overlay network may experience networking issues such as ping loss and reduced throughput. 
  
  
• BFD tunnel flapping is consistently observed for particular vmk IP address in /var/run/log/vmkernel.log

BFD_HandleStatusChange:859:[nsx@6876 comp="nsx-esx" subcomp="bfd"]local: <local-vmk-ip>, remote: <remote-vmk-ip>, oldState: up, newState: down, diag: Neighbor Signaled Session Down, type: overlay
BFD_HandleStatusChange:859:[nsx@6876 comp="nsx-esx" subcomp="bfd"]local: <local-vmk-ip>, remote: <remote-vmk-ip>, oldState: down, newState: up, diag: No Diagnostic, type: overlay
BFD_HandleStatusChange:859:[nsx@6876 comp="nsx-esx" subcomp="bfd"]local: <local-vmk-ip>, remote: <remote-vmk-ip>, oldState: up, newState: down, diag: Neighbor Signaled Session Down, type: overlay
BFD_HandleStatusChange:859:[nsx@6876 comp="nsx-esx" subcomp="bfd"]local: <local-vmk-ip>, remote: <remote-vmk-ip>, oldState: down, newState: up, diag: No Diagnostic, type: overlay


• The mapping vmnic showing UP constantly at the same time in /var/run/log/vmkernel.log

netschedHClk: NetSchedHClkNotify:4654: vmnicX: link up notification

Environment

VMware NSX
VMware vSphere ESXi

Cause

The issue is typically caused by a faulty vmnic, which results in the corresponding vmk VTEP (Virtual Tunnel Endpoint) flapping.

Such flapping, in turn, leads to network interruptions for the VMs.

Resolution

To resolve this issue, contact the hardware vendor (both for the server and the physical switch) to further troubleshoot and identify the root cause of the vmnic fault. 

Additional Information

Faulty cable could also lead into this situation. In such situations, replacing the cable is recommended.