HCX Bulk Migration fails with error "relocate on source side" in the Switchover tasks of the migration

Products

• VMware HCX

Issue/Introduction

• When attempting a Bulk Migration, it fails during the subtask of "relocate on source side".
• Login to the HCX manager as user admin and navigate to the log file /common/logs/admin/app.log, to find the entries similar to as shown below

[VmotionService_SvcThread-####, Ent: HybridityAdmin, , TxId: ######-####-####-####-###########] ERROR c.v.h.s.v.j.StartSourceSideRelocateVmWorkflow- [migId=######-####-####-####-###########] Source side relocate failed for the virtual machine. error is Currently connected network interface 'Network adapter X' uses network 'DVSwitch: ## ## ## ## ## ## ## ## ## ## ## ## ##', which is not accessible. 
com.vmware.vim.binding.vim.fault.CannotAccessNetwork: Currently connected network interface 'Network adapter X' uses network 'DVSwitch: ## ## ## ## ## ## ## ## ## ## ## ## ##', which is not accessible.

• Login to the vCenter server as user root and run the command shell.set --enabled true to access the bash shell
• Navigate to the log file location /var/log/vmware/vpxd/vpxd.log and you will see the error below

error vpxd[####] [Originator@#### sub=Default opID=OpId-######-##-##] [VpxLRO] -- ERROR lro-#####-####-####-######-######) -- vm-## -- vim.VirtualMachine.relocate: :vim.fault.CannotAccessNetwork
--> Result:
--> (vim.fault.CannotAccessNetwork) {
--> faultCause = (vmodl.MethodFault) null,
--> faultMessage = <unset>,
--> device = "Network adapter X",
--> backing = "DVSwitch[ce ## ## ## ## ## ## ## ## ## ## ## ## ## ##] NSX port group [dvportgroup-##](nsxa down)",
--> connected = true,
--> network = 'vim.dvs.DistributedVirtualPortgroup:########-####-####-####-###########:dvportgroup-##'
--> msg = ""

Environment

VMware HCX
VMware vCenter

Cause

This is caused by a stale vCenter DB entry. 

Resolution

Need to ensure that the stale entry of the VM DVport is changed or reset in the vCenter Database.

• Resolution 1 :-
  • Migrate the virtual machine to the standard switch and migrate the VM back to the DVS switch.
    
    
• Resolution 2 :- 
  • Migrate the virtual machine to any other available cluster and ensure the network for the VM is selected on a different DVS witch.
  • Migrate the VM back to the previous original cluster on the original DVS switch. This will force vCenter to update the database with a new port entry
    
    
• Resolution 3 :- 
  • Power off the VM and remove the VM from the vCenter inventory. Re-register the VM back on the same cluster where it was registered earlier. This will ensure that the VM DVport ID is changed.

Additional Information

Follow the article here to know How to manually clean up unused or stale Distributed Virtual Switch (VDS) on an ESXi host