Virtual machine network connectivity loss after vMotion or HA failover

Products

• VMware vSphere ESXi
• VMware Cloud Foundation

Issue/Introduction

• Virtual machines experience total network isolation following migration or failover.
• Guest OS remains unreachable via ICMP or application protocols.
• Toggling the virtual network adapter state does not restore connectivity.
• Adding a dummy vNIC restores connectivity immediately.
• Packet captures show RARP packets egressing ESXi but not being processed by the physical switch.
• Setting the Load Balancing policy in the vSwitch or the Teaming and failover policy in the Distributed Port Group properties to Notify Switches, the host sends the RARP request after performing a vMotion.

Environment

ESX: 8.0 Update 3
VCF: 5.2

Cause

The network outage was caused by a misconfiguration on the physical upstream switch port connected to the vmnic the VM is using.

Resolution

1. Verify "Notify Switches" is set to Yes in the Load Balancing or Teaming and Failover policy of the Port Group.
2. Ensure the virtual machine is on the proper VLAN ID by checking the port configuration of the vSwitch and physical switch.
3. Check the VLAN configuration on the vSwitch.   Note: If a physical switch port is not set to PortFast or Edge Port, connectivity will be blocked during STP convergence after a vMotion.
4. Check the subnets assigned to VLANs.
5. Check the physical switch ports for proper VLAN configuration.
6. Check the MAC table of the upstream switch is properly updating.

Additional Information

For more information on troubleshooting steps, see Troubleshooting virtual machine network connection issues.

For further information on packet captures on ESXi, see Packet capture on ESXi using the pktcap-uw tool.

To perform packet captures with pktcap-uw:

• Identify the VM Switchport and Uplink:
  Locate the VM's active Switchport ID and the physical vmnic it utilizes on the destination host:

  net-stats -l | grep <VM_Name>



• Capture VM ARPs (VnicTx/VnicRx):
  pktcap-uw --switchport <Switchport_ID> --capture VnicTx,VnicRx --mac <VM_MAC_Address> --ng -o /tmp/vm_vnic_arp.pcapng

• Capture Uplink RARPs (UplinkSndKernel):
  After a vMotion operation, the receiving host sends a RARP packet on behalf of the VM to update the rest of the network of the VM's new location.

• Identify which uplink is used by the VM:
  Type esxtop and press n for network.

• Capture directly on the uplink to verify this egress traffic:
  pktcap-uw --uplink <vmnic_name> --capture UplinkSndKernel --mac <VM_MAC_Address> --ng -o /tmp/uplink_vmnic_rarp.pcapng