vMotion takes a long time to complete

Products

• VMware vSphere ESXi

Issue/Introduction

• Migrating a VM between two identical hosts takes a long time to complete.
• There are no underlying performance issues with CPU, Memory and Storage. 
• Used the esxtop command to monitor network statistics during the vMotion process.
• No packet drops observed for the vMotion vmkernel and the associated network adapter during vMotion.
  
  • Note: Please monitor the stats of the network adapter stats by running the command: esxcli network nic stats get -n vmnicX
• Validated network bandwidth availability between the source and destination ESXi hosts. Bandwidth is sufficient for vMotion operations.
• In the VMs vmware.log, the task has progressed to the PRECOPY stage,

vmx - MigrateSetState: Transitioning from state MIGRATE_FROM_VMX_WAITING (9) to MIGRATE_FROM_VMX_PRECOPY (10)


Environment

VMware vSphere ESXi

Cause

There are Receive CRC errors on the active physical adapter used for vMotion.

CRC stands for Cyclic Redundancy Check, a method used to detect errors in data transmission. When the source host creates a packet, it calculates the CRC value using a specific algorithm. This calculated CRC value is then placed in the FCS field and sent along with the packet. When the destination host receives the packet, it performs the same CRC calculation on the received data. If the result matches the CRC value in the FCS field, the packet is considered error free and is processed as expected. If it doesn’t match, the packet is assumed to be corrupted during transmission and is discarded, this is known as a CRC Error.

Resolution

To troubleshoot vmnic errors reported on ESXi, please refer: Troubleshooting NIC errors and other network traffic faults in ESXi

To continuously check for increase in packet errors on a physical uplink of ESXi host, use the below command:

watch -d esxcli network nic stats get -n vmnicX

Additional Information

• Please refer to this article for detailed information regarding vMotion process: The vMotion Process Under the HoodRunning 
• To monitor CPU performance, please refer article: Troubleshooting ESXi performance issues
• For more details on using esxtop to identify storage issues, please refer: Using esxtop to identify storage performance issues for ESXi
• For details on performing iperf tests please refer article: How to use iPerf to test bandwidth between source host and target