Virtual machines been stuck at the base sync stage for multiple days without any progress.
search cancel

Virtual machines been stuck at the base sync stage for multiple days without any progress.

book

Article ID: 429396

calendar_today

Updated On:

Products

VMware HCX VMware vSphere ESXi

Issue/Introduction

  • VM replication tasks remain in progress without completion.
  • Bulk migration does not proceed been stuck at the base sync:

  •  ESXi host : var/run/log/vmkernel.log

ScsiDeviceIO: 3511: Cmd(#####40a26780) 0x28, CmdSN ####fc54 from world ####216 to dev "naa.###############" failed H:0x0 D:0x2 P:0x0 Valid sense data: 0x3 0x11 0x0.
WARNING: Hbr: 1303: Failed to read checksum data (DiskID=RDID-05d7d###-XXXb-4##d-bee0-816b8d49d###) (offset35888###960) (extentSize=8192) (count=8): I/O error
ScsiDeviceIO: 3511: Cmd(0x####55626800) 0x28, CmdSN 0xc###c44 from world 2###216 to dev "naa.###################" failed H:0x0 D:0x2 P:0x0 Valid sense data: 0x3 0x11 0x0.

or 

WARNING: Hbr: 1559: Failed to read checksum data (DiskID=RDID-###################) (offset10195435520) (extentSize=8192) (count=8):Checksum mismatch

Environment

VMware HCX

Cause

An unrecovered read error (bad disk sector) on the underlying physical storage device hosting the impacted VM's datastore prevents the ESXi Host-Based Replication (HBR) engine from reading disk blocks and checksum data, causing the HCX bulk migration base sync operation to stall indefinitely.

Resolution

  1. Cancel the Migration Task:
    • Log in to the VMware HCX Manager UI, navigate to Services > Migration > Migration Tracking, and cancel or abort the stalled bulk migration task so the VM files unlock.
  2. Relocate the VM to a Different Datastore and Retry the Migration:
    • In the vSphere Client, identify a healthy, accessible datastore not residing on the failing naa.6* storage device.
    • Right-click the impacted VM, select Migrate > Change storage only (Storage vMotion), and move the VM files to the healthy datastore.
    • Once Storage vMotion completes, return to the VMware HCX Manager UI and re-initiate the bulk migration.
  3. Investigate Storage Corruption if Relocation Fails:
    • If the Storage vMotion relocation fails with similar I/O or SCSI read errors, engage your storage team or hardware vendor to investigate and remediate the underlying storage corruption.
  4. Alternative Migration Method:
    • You can also use VMware vCenter Converter to perform a Virtual-to-Virtual (V2V) migration of the impacted VM.

Additional Information

Unable to vMotion, clone, backup, or consolidate VMs on vSAN error Checksum mismatch - VMware vSAN