HCX RAV fails during switchover with "Could not copy complete network copy for file [Datastore][VM-name]/[VM-name].nvram]"
search cancel

HCX RAV fails during switchover with "Could not copy complete network copy for file [Datastore][VM-name]/[VM-name].nvram]"

book

Article ID: 449118

calendar_today

Updated On:

Products

VMware HCX

Issue/Introduction

Replication Assisted vMotion (RAV) migrations fail intermittently during the final switchover phase. The HCX manager UI and logs report a generic system error.

Symptoms:

  • Migrations successfully complete initial synchronization but fail during the vMotion cutover.
  • Error in HCX UI:
    vMotion failed. System Error. Source side error is : Source side relocate failed for the virtual machine. A specified parameter was not correct: Target side error is : Could not complete network copy for file [Datastore] [VM-name]/[VM-name].nvram
  • The Source HCX app.log contains the following error: 
    ERROR c.v.h.s.v.j.MonitorSourceSideProgressWorkflow - Source side relocate 'task-ID' failed for the virtual machine. Error is A specified parameter was not correct: . Total progress % is 'null'. com.vmware.vim.binding.vmodl.fault.InvalidArgument: A specified parameter was not correct

     

  • Target site app.logs would show failure during the NFC copy of the VM state.
    2026-06-02T05:47:28.004Z ERROR hcx-app-engine 31## [hcx@4413 logger="c.v.h.s.v.j.MonitorTargetSideProgressWorkflow" thread="VmotionService_SvcThread-#####" ent="Ent: HybridityAdmin" user="" txid="TxId: ####-a98c-####-8bfd-#######"] [migId=######-####-####-####-######] Target side relocate 'task-26527' failed for the virtual machine. Error is Could not complete network copy for file [Datastore] [VM-name]/[VM-name].nvram. Total progress % is null.
com.vmware.vim.binding.vim.fault.NetworkCopyFault: Could not complete network copy for file [Datastore] [VM-name]/[VM-name].nvram
  • Re-running the migration often completes successfully.
  • Source ESXi vmkernel.log logs show the vMotion stream successfully binding between the source host and the HCX-IX appliance before hitting the throughput bottleneck:

    vMotion stream starts between Source ESXi and IX Appliance:

    cpu36:28240916)Migrate: 102: <Migration-ID> S: MigrateState: Starting migration off
cpu61:28242631)MigrateNet: 1753: <Migration-ID> S: Successfully bound connection to vmknic vmk1 - '<Source-vMotion-IP>'
cpu52:2098573)MigrateNet: vm 2098573: 3232: Accepted connection from <IX-vMotion-IP>
cpu61:28242631)XVMotion: 3379: <Migration-ID> S: Starting XVMotion stream.

    Bandwidth fluctuations observed during the transfer:

    cpu68:28240926)VMotion: 4903: <Migration-ID> S: Estimated network bandwidth 2.398 MB/s during disk copy
cpu19:28240921)VMotion: 4851: <Migration-ID> S: Another pre-copy iteration needed ... (network bandwidth ~164.951 MB/s)
cpu60:28242628)VMotion: 2199: <Migration-ID> S: VMotion bandwidth in last 1s: 128 MB/s, 10s: 162 MB/s, 20s: 167 MB/s

Environment

VMware HCX 9.x

Cause

This failure is usually triggered by unstable network throughput on the vMotion link between the source host and the HCX-IX appliance.

When HCX starts the switchover, it has to move the VM’s configuration files (like .nvram and .vmx) using the NFC protocol. If the bandwidth fluctuates and drops below the 150 Mbps threshold, the file transfer can't complete in time. You’ll see the connection time out in the vmkernel.log, often preceded by entries showing the throughput bottoming out. Since these small files never arrive, HCX can't move forward with the switchover.

Resolution

This is an environmental networking issue external to the HCX application. To resolve the issue:

  1. Investigate Physical Networking: Engage the physical networking team to identify and remediate the source of bandwidth fluctuations and congestion between the source ESXi management/vMotion interfaces and the HCX Interconnect appliances.
  2. Verify Bandwidth Requirements: Ensure the environment consistently meets the minimum requirement of 150 Mbps for vMotion/RAV operations.
  3. Optimize Network Path: Confirm that vMotion traffic has dedicated physical ports and, where possible, eliminate Layer 3 (L3) routing hops by placing IX appliances and ESXi hosts on the same Layer 2 (L2) network.

Workaround: If the network cannot be immediately stabilized, use the following technical paths:

  1. Throttling: Reduce the number of simultaneous migrations in a single wave to effectively utilize available bandwidth.
  2. Alternative Migration Method: Use Bulk Migration. Unlike RAV, Bulk Migration facilitates concurrent transfers and is more resilient to intermittent network slowness as it does not require a live vMotion state transfer.
  3. Method Selection: Reserve RAV only for mission-critical VMs that strictly require zero downtime.

Additional Information

HCX Replication Assisted vMotion (RAV) migrations fail with " Source side relocate failed for the virtual machine" in HCX 9.0.0

 

Powered by Wolken Software