Virtual Machine Network Connectivity Loss After vMotion or vSphere HA Failover
search cancel

Virtual Machine Network Connectivity Loss After vMotion or vSphere HA Failover

book

Article ID: 318873

calendar_today

Updated On:

Products

VMware vSphere ESXi VMware Cloud Foundation

Issue/Introduction

  • Virtual machines (VMs) lose network connectivity immediately following a vMotion migration or vSphere HA failover event.

  • The guest OS fails to respond to ICMP ping or incoming application traffic.

  • Disconnecting and reconnecting the virtual network adapter (vNIC) inside the guest OS does not restore network connectivity.

  • Adding a secondary dummy vNIC to the VM temporarily restores network connectivity on the original interface.

  • Packet captures confirm Reverse ARP (RARP) packets egress the physical ESXi uplink (vmnic), but the physical switch MAC address table does not update to reflect the VM's new physical port location.

Environment

  • VMware vSphere ESXi 8.0 Update 3

  • VMware Cloud Foundation (VCF) 5.2

  • vSphere Distributed Switch (VDS)

Cause

The upstream physical switch does not process or blocks Gratuitous ARP (GARP) and Reverse ARP (RARP) packets transmitted by ESXi upon migration.

Consequently, the physical switch continues forwarding traffic to the source ESXi host's physical switch port rather than relearning the VM MAC address on the destination host's switch port.

This issue can also occur if the upstream switch port lacks Spanning Tree Protocol (STP) PortFast/Edge Port configuration, causing port blocking during topology transitions.

Resolution

1. Verify vMotion State on Destination ESXi Host

Confirm that the virtualization layer completed the migration successfully:

  1. Open an SSH session to the destination ESXi host.
  2. Inspect the virtual machine log:

    grep "MigrateSetStateFinished"/vmfs/volumes/<datastore>/<vm_name>/vmware.log
  3. Confirm the presence of the following entry:

    MigrateSetStateFinished: type=2 new state=MIGRATE_FROM_VMX_FINISHED

2. Verify Distributed Port Group "Notify Switches" Setting

  1. Log in to the vSphere Client.

  2. Navigate to Inventory > Networking > select your Distributed Switch.

  3. Right-click the target Distributed Port Group and select Edit Settings.

  4. Go to Teaming and failover.

  5. Ensure Notify switches is set to Yes.

3. Verify Physical Switch Configuration

Coordinate with your network engineering team to validate the following physical switch settings:

  • GARP/RARP Filtering: Ensure upstream switch ports do not drop, rate-limit, or inspect-block GARP/RARP broadcast frames (such as overly restrictive Dynamic ARP Inspection or Storm Control policies).

  • VLAN Trunking: Verify the port group VLAN ID is consistently allowed and trunked on all physical switch ports connected to both the source and destination ESXi uplinks.

  • Spanning Tree Protocol (STP): Ensure all ESXi-facing switch ports have STP PortFast (Cisco) or Edge Port (Arista/Juniper) enabled to prevent the port from entering temporary listening/learning states during MAC move events.

4. Monitor MAC Address Learning

  1. On the upstream physical switch CLI, query the MAC address table for the VM's MAC address before initiating migration.

  2. Initiate a vMotion migration of the VM to the destination ESXi host.

  3. Query the MAC address table again and confirm that the association successfully updates from the source host's physical switch port to the destination host's physical switch port.

Diagnostic Procedures

If the issue persists, capture network traffic on the destination ESXi host to verify packet transmission.

Identify VM Switchport ID and Uplink

  1. Obtain the VM switchport ID and assigned uplink name:

    net-stats -l | grep "<VM_Name>"

  2. Note the returned PortNum and corresponding vmnic interface.

Capture VM vNIC Traffic

Run pktcap-uw to capture ingress and egress traffic at the virtual NIC level:

pktcap-uw --switchport <Switchport_ID> --capture VnicTx,VnicRx --mac <VM_MAC_Address> --ng -o /tmp/vm_vnic_arp.pcapng

Capture Uplink RARP Broadcasts

Verify that ESXi successfully generates and transmits the RARP announcement out of the physical adapter:

pktcap-uw --uplink <vmnic_name> --capture UplinkSndKernel --mac <VM_MAC_Address> --ng -o /tmp/uplink_vmnic_rarp.pcapng

Review the resulting .pcapng files in Wireshark to confirm packet egress from the ESXi host.

Additional Information

For more information on troubleshooting steps, see Troubleshooting virtual machine network connection issues.

For further information on packet captures on ESXi, see Packet capture on ESXi using the pktcap-uw tool.

To perform packet captures with pktcap-uw:

  • Identify the VM Switchport and Uplink:
    Locate the VM's active Switchport ID and the physical vmnic it utilizes on the destination host:
    net-stats -l | grep <VM_Name>

  • Capture VM ARPs (VnicTx/VnicRx):
    pktcap-uw --switchport <Switchport_ID> --capture VnicTx,VnicRx --mac <VM_MAC_Address> --ng -o /tmp/vm_vnic_arp.pcapng

  • Capture Uplink RARPs (UplinkSndKernel):
    After a vMotion operation, the receiving host sends a RARP packet on behalf of the VM to update the rest of the network of the VM's new location.

  • Identify which uplink is used by the VM:
    Type esxtop and press n for network.

  • Capture directly on the uplink to verify this egress traffic:
    pktcap-uw --uplink <vmnic_name> --capture UplinkSndKernel --mac <VM_MAC_Address> --ng -o /tmp/uplink_vmnic_rarp.pcapng