VMs lose network connectivity after vMotion/migration due to incorrect uplink teaming configuration

Products

• VMware vSphere ESXi

Issue/Introduction

• VMware vSphere Virtual Machines (VMs) randomly lose network connectivity during or immediately after a vMotion migration between ESXi hosts.
• Disabling and re-enabling the virtual Network Interface Card (vNic) occasionally restores network connectivity.
• A review of the dvSwitch ports reveals that the affected VM portgroups are improperly configured to use all available uplinks (Uplinks 1-4) as Active, conflicting with storage-dedicated uplinks.

For example: 

• vmnic1 was on uplink2
• vmnic2 was on uplink4

NOTE: Changing the teaming to the image resolved this particular issue.

Environment

• VMware vSphere ESXi - ANY
• VMware vCenter Server - ANY

Cause

Scenario 1: The VM network portgroups are configured with all four uplinks set to "Active." However, uplinks 3 and 4 are dedicated strictly to storage traffic and are not trunked for virtual machine VLAN traffic. Following a vMotion, the virtual switch load balancing policy hashes the VM's MAC address to egress on either uplink 3 or 4, resulting in a Layer 2 traffic blackhole and immediate connectivity loss.

Scenario 2: Mixed combination of Active Unused with the incorrect ports so that 1 Data uplink is teamed with 1 VSAN uplink causing issues for the teaming algorithm and subsequently causing intermittent storage and VM network data loss.

This also explains why disconnecting and reconnecting the vNic occasionally restores connectivity, as this operation will initiate a new hash calculation for NIC teaming, potentially causing the VM to be mapped to an uplink configured for VM traffic.

Resolution

• Log in to the vSphere Client.
• Navigate to the Networking view and select the affected Distributed Virtual Switch.
• Right-click the impacted VM portgroup and select Edit Settings.
• Navigate to Teaming and failover.
• Under the Failover order, select the storage-dedicated uplinks (e.g., Uplink 3 and Uplink 4) and move them down to the Unused uplinks category.
• Click OK to save the configuration.
• Repeat this process for all VM portgroups that are improperly spanning the storage-dedicated uplinks.

Additional Information

VMs intermittently lose network connections after vMotion