Intermittent network connectivity observed between a VM on ESXi host with external server

Products

• VMware vSphere ESXi

Issue/Introduction

The intermittent network connectivity is observed only between a VM on ESXi hosts and external server 

ARP requests for the VM subnet xx.xx.xx.xx don't arrive at the VM. when a ping is done from the VM the physical infrastructure (ACI) learns the MAC, but it ages out in 30 minutes

Below actions were performed but it did not resolve the issue  

• Migrated the vm to other host
• Removed old NIC of VM and added new NIC on vcenter and did re-configuration
• Performed vNIC reset of VM

Environment

8.0.3 Build : 24585383

Cause

There were two issues on the setup as below 

1. From net-stats-l o/p from ESXi hosts MAC address for vnic is mentioned as 00:00:00:00:00:00 as below 

But the MAC address is assigned correctly on vcenter for the interface of VM 

When a virtual machine's network adapter is disabled within the guest operating system, the MAC address displayed in ESXi might show as 00:00:00:00:00:00 (all zeros).

2. The MAC aging timer on ACI switches connected to the uplinks via PE router is 30 min. After 30 min of inactivity of the VM, the MAC address that is learned on ACI gets cleared. So when there is an incoming packet to the VM inside ESXi hosts, its not switched or forwarded to next hop. 

Resolution

• The guest operating system manages the network interface's status (enabled/disabled).
  
  
• When disabled, the guest OS doesn't report any MAC address to the ESXi host. In the guest operating system, enable the virtual network adapter that will allow the guest OS to determine a MAC address, and that MAC address will be reported back to ESXi. 

Refer KB 370369 for more information 

To address the issue of clearing VM MAC address on ACI, check with VNF vendor to implement feasible solution of updating VNF MAC address to ACI periodically so that VM MAC address is learned periodically and retained on ACI.