vMotion fails with the error: Migration to host <> failed with error

Products

• VMware vSphere ESXi

Issue/Introduction

vMotion fails with Already disconnected or Timeout errors.

• Example 1:
  
  Migration to host <> failed with error Already disconnected (195887150).
VMotion [168362677:1280508768956681] write function failed.
VMotion [168362677:1280508768956681] failed to flush channel: Already disconnected
VMotion [168362677:1280508768956681] socket connected returned: Already disconnected
   
• Example 2:
  
  Migration to host <> failed with error Timeout (195887137).
VMotion [-1408234551:1280495981317190] write function failed.
VMotion [-1408234551:1280495981317190] failed to send final vbuf: Timeout
VMotion [-1408234551:1280495981317190] timed out waiting 20002 ms to transmit data.
   
• Example 3:
  
  Migration to host <> failed with error Timeout (195887137).
VMotion [171115571:1280406775498450] write function failed.
VMotion [171115571:1280406775498450] failed to flush channel: Timeout
VMotion [171115571:1280406775498450] timed out waiting 20001 ms to transmit data.

Environment

VMware ESXi 7.0

Cause

The errors suggest that the destination host failed the vMotion prior to completion, which is likely caused by network issues.

Resolution

To check the health of the vMotion network:

• Check for IP address conflicts on the vMotion network. Each host in the cluster should have a vMotion vmknic, assigned a unique IP address.
• Check for packet loss over the vMotion network. Try having the source host ping (vmkping) the destination host's vMotion vmknic IP address for the duration of the vMotion.
• Check for the ability to pass traffic over the vMotion network. Consider whether jumbo frames are enabled on the VMkernel port, NIC, and Switch. Consider whether a firewall may be blocking vMotion traffic.

If the preceding diagnostic checks reveal an issue, apply the appropriate remedies. If necessary, engage your network team or your network vendor, as appropriate.

It is rare but possible that the source waited too long between page transmissions to the destination host. This can occur, for example, if there is a pending question for the virtual machine that requires user interaction prior to being able to migrate the virtual machine. Check that the virtual machine is running and responsive to input and that it is not waiting for administrator interaction in vCenter Server. Remediate any issue you discover here.

 

Alternatively, if the source host is heavily over committed, with slow or failing storage such that swap I/O could be delayed significantly (minutes per I/O), the destination may give up on the vMotion. This case is extremely rare and is visible mainly during storage failovers. If this occurs, retry the operation when the source host is less overloaded. There may also be an issue with heavy latency in the storage array, and you can engage your storage team or your storage vendor to diagnose and fix the issue.

Additional Information

Impact:
Network packet loss can cause random vMotion failures, significantly impacting vMotion's ability to succeed reliably. Retrying the vMotion operation may be successful, but VMware recommends that you follow this article to isolate and correct the problem.