Virtual Machine Network Connectivity Loss After vMotion or HA Failover

Products

• VMware vSphere ESXi
• VMware Cloud Foundation

Issue/Introduction

Virtual Machines experience network connectivity loss or unresponsiveness immediately following a vMotion migration or HA failover.

The guest OS remains unreachable via ICMP or application protocols. Toggling the virtual network adapter state does not restore connectivity, but adding a dummy vNIC restores connectivity immediately.

Packet captures show RARP packets egressing ESXi but not being processed by the physical switch. This behavior correlates with the physical switch failing to update MAC address tables or ARP caches.

Environment

VMware vSphere ESXi 8.0 Update 3
VMware Cloud Foundation (VCF) 5.2

Cause

A misconfiguration on the physical upstream switch port connected to the active vmnic prevents Gratuitous ARP (GARP) and Reverse ARP (RARP) packets from updating MAC address tables.

Resolution

1. Validate the virtualization layer migration completed successfully by confirming the migration state in the vmware.log on the destination ESXi host. Look for: MigrateSetStateFinished: type=2 new state=MIGRATE_FROM_VMX_FINISHED.

2. Verify "Notify Switches" is set to "Yes" in the Load Balancing or Teaming and Failover policy of the Distributed Port Group.

3. Ensure the upstream physical switch permits Gratuitous ARP (GARP) and Reverse ARP (RARP) packets.

4. Monitor MAC address learning during a test migration by tracing the MAC address table on the physical switch. Verify that the association updates from the source host port to the destination host port.

5. Confirm that the VLAN associated with the VM exists, is assigned the correct subnet, and is correctly trunked on the destination physical switch ports and vSwitch. Note: If a physical switch port is not set to PortFast or Edge Port, connectivity will be blocked during Spanning Tree Protocol (STP) convergence after a vMotion.

6. If the issue persists, gather VMware vCenter and ESXi Support Bundles and perform a packet capture on the vmnic of the destination host to verify packet transmission.

Additional Information

For more information on troubleshooting steps, see Troubleshooting virtual machine network connection issues.

For further information on packet captures on ESXi, see Packet capture on ESXi using the pktcap-uw tool.

To perform packet captures with pktcap-uw:

• Identify the VM Switchport and Uplink:
  Locate the VM's active Switchport ID and the physical vmnic it utilizes on the destination host:
  net-stats -l | grep <VM_Name>

• Capture VM ARPs (VnicTx/VnicRx):
  pktcap-uw --switchport <Switchport_ID> --capture VnicTx,VnicRx --mac <VM_MAC_Address> --ng -o /tmp/vm_vnic_arp.pcapng

• Capture Uplink RARPs (UplinkSndKernel):
  After a vMotion operation, the receiving host sends a RARP packet on behalf of the VM to update the rest of the network of the VM's new location.

• Identify which uplink is used by the VM:
  Type esxtop and press n for network.

• Capture directly on the uplink to verify this egress traffic:
  pktcap-uw --uplink <vmnic_name> --capture UplinkSndKernel --mac <VM_MAC_Address> --ng -o /tmp/uplink_vmnic_rarp.pcapng