Virtual Machines experience network connectivity loss after vMotion

Products

• VMware vSphere ESXi

Issue/Introduction

After migrating a Virtual Machine (VM) to a destination ESXi host via vMotion, the VM experiences a loss of network connectivity.

Environment

VMware ESXi

Cause

The upstream physical network switch ports connected to the ESXi host where the destination VM is running are incorrectly configured or incomplete. Missing configuration details, such as omitted VLANs on a trunk port, prevent the Layer 2 broadcast traffic (ARP) from reaching the Default Gateway, isolating the VM at Layer 2.

A packet capture will result in repeated ARP requests when the VM is trying to reach the Default Gateway.

Resolution

• Identify the destination ESXi host where the VM resides and determine the active physical uplinks (vmnics) associated with the affected virtual switch or distributed switch.
• Engage the physical network administration team to inspect the upstream physical switch ports connected to the identified vmnics.
• Verify and correct the switch port configuration to ensure all required VLANs are permitted and 802.1Q trunking is properly established.
• Confirm network restoration by verifying successful ARP resolution and outbound ICMP/TCP communication from the affected VM.

Additional Information

To validate the condition locally on the ESXi host before engaging network teams, the pktcap-uw utility can be used on the ESXi command line to observe the outbound ARP broadcasts and the explicit lack of ingress ARP replies.

Example:

pktcap-uw --uplink vmnic<#> --capture UplinkSndKernel -o - |tcpdump-uw -r - -n host <Default Gateway_IP>

Where <#> is the number of the physical uplink found using: esxtop --> n (n keyboard character) on the ESXi host where the destination VM is running

 

Please reference the following KB for using the pktcap-uw tool

Packet capture on ESXi using the pktcap-uw tool