Error: Migration to host failed with error Connection closed by remote host during HCX RAV
search cancel

Error: Migration to host failed with error Connection closed by remote host during HCX RAV

book

Article ID: 451005

calendar_today

Updated On:

Products

VMware HCX

Issue/Introduction

Consistent Replication Assisted vMotion (RAV) migrations of workload VMs fail when targeting specific ESXi hosts within a destination site cluster.

  • The migration fails at Step 44 - Relocate task failed on source site.
  • When retrying the migration to the affected ESXi host, the process hangs at Step 43 - Starting relocate task on source side before eventually failing at Step 44.
  • Migrating the same workloads to neighboring ESXi nodes in the cluster is successful.
  • Service mesh diagnostics report no issues (all required ports connect successfully).
  • Transport Analytics shows that connection is health between service meshes (high throughput, no latency, no drops).
  • The following vMotion network timeout errors are found in /common/logs/admin/app.log:
    [VmotionService_SvcThread-13156, Ent: HybridityAdmin, , TxId: ########-####-####-####-############] ERROR c.v.h.s.v.j.MonitorSourceSideProgressWorkflow- [migId=########-####-####-####-############] Source side relocate 'task-4886527' failed for the virtual machine. Error is Migration to host <ESXi IP> failed with error Connection closed by remote host, possibly due to timeout (195887167).  msg.checkpoint.precopyfailure:Migration to host <ESXi IP> failed with error Connection closed by remote host, possibly due to timeout (195887167).  vob.vmotion.net.send.start.failed:vMotion migration [181146722:2634356530565833181] failed to send init message to the remote host <ESXi IP> vob.vmotion.net.rcv.failed.status.type:vMotion migration [181146722:2634356530565833181] (0-73777998199648) failed to receive 68/68 bytes from the remote host <ESXi IP>: Connection closed by remote host, possibly due to timeout faultTime:<Fault Date>. Total progress % is 'null'.
  • Manually vMotioning a VM from healthy ESXi host to the affected ESXi host in the same cluster is giving an error:
    "Currently connected network interface" 'Network adapter X' uses network 'DVSwitch[50 3e ## ## ## ## ## ##-## ## ## ## ## ## 42 4e] NSX port group [dvportgroup-#####](nsxa down)', which is not accessible."

Environment

  • VMware HCX
  • VMware NSX
  • VMware ESXi

Cause

  • The destination ESXi host is not in healthy configuration state with NSX.
  • Because the ESXi host is in a bad state with NSX, HCX is able to complete the initial replication phase but fails immediately when attempting to initiate the VM via the vMotion task on the target host.
  • This causes the initialization message to fail to send to the remote host, leading to the connection timeouts observed in the logs.

Resolution

To resolve this issue, you must restore the health of the affected ESXi host within NSX:

  1. Log in to the NSX Manager UI and navigate to the fabric/host transport nodes view to confirm the failed state of the target ESXi host.
  2. Investigate and resolve the underlying issue causing the nsxa service to be down on the destination ESXi node. 
  3. Once NSX reports the host as healthy and the nsxa service is running, cancel the stuck migration in HCX and force a cleanup.
  4. Retry the RAV migration of the workload VMs.

Additional Information

vMotion is blocked due to error "Currently connected network interface ... uses network (nsxa down), which is not accessible"
VM Migration Fails with "Network Not Accessible" Error Due to NSX Segment Inaccessibility on Partially Configured Host
After ESXi host upgrade vMotion is blocked due to error: "Currently connected network interface uses Network .. NSX port group ..(nsxa down), which is not accessible"
Creating and managing Broadcom support cases
vMotion is blocked due to error "nsxa down"

Powered by Wolken Software