[VMC on AWS] Migrating a Virtual Machine from One Host to another fails with an error: "Operation Timed out"

Products

• VMware Cloud on AWS

Issue/Introduction

• This article provides steps to troubleshoot the error: "Operation Timed out" received while performing the Migration of a VM from one host to another.
• Migrating a Virtual Machine from One Host to another fails with an error: "Operation Timed out"
• The below entries are observed in the /var/log/hostd log on the source host:

  <99>1 YYYY-MM-DDTHH:MM:SS.sssZ vcenter vpxd 8371 - -  Event [7866265] [1-1] [YYYY-MM-DDTHH:MM:SS.sssZ] [vim.event.EventEx] [info] [] [SDDC-Datacenter] [7866265] [Alarm 'Migration error' on <vmname> triggered by event 7866259 'Cannot migrate <vmname> from xx.x.x.x, WorkloadDatastore in SDDC-Datacenter to xx.x.x.x, WorkloadDatastore in SDDC-Datacenter']
  
  <99>1 YYYY-MM-DDTHH:MM:SS.sssZ vcenter vpxd 8371 - -  Event [7866562] [1-1] [YYYY-MM-DDTHH:MM:SS.sssZ] [vim.event.EventEx] [info] [PHILIPS-CS\I-001] [SDDC-Datacenter] [7863211] [Failed to revert the execution state of the virtual machine <vmname> on host xx.x.x.x, in compute resource Cluster-1 to snapshot InitialSnapshot, with ID 1]
  
  <99>YYYY-MM-DDTHH:MM:SS.sssZ host.domain.com Hostd: warning hostd[2101667] [Originator@6876 sub=Vmsvc.vm:/vmfs/volumes/vsan:############-##########/#######-####-####-####-########/<vmname>.vmx opID=xxxxxxxxx-xxxxxx-xxxx-xxxx-xx:xxxxxxx-xx-xx-xxxx user=vpxuser:<username>] Failed to revert snapshot(s): N3Vim5Fault8Timedout9ExceptionE(Fault cause: vim.fault.Timedout
  
  <99>YYYY-MM-DDTHH:MM:SS.sssZ host.domain.com Fdm: info fdm[2105324] [Originator@6876 sub=Policy opID=yyyyyy-yyyy] Reset no 0 failed for VM /vmfs/volumes/vsan############-##########/#######-####-####-####-#######/<vmname>.vmx
  
  <99>YYYY-MM-DDTHH:MM:SS.sssZ host.domain.com Hostd: verbose hostd[2107109] [Originator@6876 sub=Vigor.Vmsvc.vm:/vmfs/volumes/vsan:############-##########/#######-####-####-####-#######/<vmname>-1.vmx] Get vigor fields VigorCpp::Devices::Migrate::Online translated error to vim.fault.InvalidState
  
  <99>YYYY-MM-DDTHH:MM:SS.sssZ host.domain.com Hostd: info hostd[2101428] [Originator@6876 sub=Vmsvc.vm:/vmfs/volumes/vsan:############-##########/#######-####-####-####-#######/<vmname>.vmx opID=yyyyyy-yyyy user=vpxuser] Error retrieving file layout : N3Vim5Fault8Timedout9ExceptionE(Fault cause: vim.fault.Timedout

Cause

Virtual Machine (VM) may have locks on its files, or it could be left in a zombie state i.e. The VM is unregistered from the VMC vCenter.

Resolution

Perform the below steps to resolve the Migration Failure issue: 

1. Power off the VM
2. Consolidate All snapshots present on the VM.
3. Attempt the Migration again from the powered-off state.
4. If the Migration succeeds, then ensure that the VM is powered On and attempt a test with the original Migration operation again.

Additional Information

Migrate a Virtual Machine to a New Host and Datastore by Using vMotion in the vSphere Web Client

Impact/Risks:
Unable to perform any actions on the VM.