After upgrading ESXi 8.0, VMs protected by Veeam CDP fail to vMotion and hang.

Products

• VMware vSphere ESXi 8.0

Issue/Introduction

In the vmware.log, there are records indicating CBT disconnection and vMotion timeout events.

 

YYYY-MM-DDTHH:MM:SS.###Z In(05) vcpu-0 - MigrateSetState: Transitioning from state MIGRATE_TO_VMX_PRECOPY (3) to MIGRATE_TO_VMX_CHECKPT (4).
YYYY-MM-DDTHH:MM:SS.###Z In(05) vcpu-0 - Migrate: Preparing to suspend.
YYYY-MM-DDTHH:MM:SS.###Z In(05) vcpu-0 - Migrate: VM starting stun, waiting 100 seconds for go/no-go message.
……
YYYY-MM-DDTHH:MM:SS.###Z In(05) vcpu-0 - Closing disk 'scsi0:2'
YYYY-MM-DDTHH:MM:SS.###Z In(05) vcpu-0 - DISKLIB-CBT   : Shutting down change tracking for untracked fid 2913320.
YYYY-MM-DDTHH:MM:SS.###Z In(05) vcpu-0 - DISKLIB-CBT   : Successfully disconnected CBT node.

YYYY-MM-DDTHH:MM:SS.###Z In(05) vmx - MigrateSetState: Transitioning from state MIGRATE_FROM_VMX_WAITING (9) to MIGRATE_FROM_VMX_PRECOPY (10).
YYYY-MM-DDTHH:MM:SS.###Z In(05) worker-3744960 - MigrateBusMemPrealloc: BusMem preallocation begins.
YYYY-MM-DDTHH:MM:SS.###Z In(05) worker-3744960 - MigrateBusMemPrealloc: BusMem preallocation completes.
YYYY-MM-DDTHH:MM:SS.###Z In(05) vmx - MigrateWaitForData: Waited for 183.12 seconds.
YYYY-MM-DDTHH:MM:SS.###Z Wa(03) vmx - Migrate: timed out waiting for data from the source.

 

We can see "veecdp" within the VMDK file.

ddb.iofilters = "veecdp"
ddb.longContentID = "************************"
ddb.sidecars = "veecdp_1,************************.vmfd"

 

In veedcp.log which show:

YYYY-MM-DDTHH:MM:SS.###Z In(158) veecdp[2111649]: [0002111824] 914 (I) [WorkGroup] WorkGroup wait started, self: 0xfcdb06ca40
YYYY-MM-DDTHH:MM:SS.###Z In(158) veecdp[2111649]: [0002111824] 914 (I) [WorkGroup] WorkGroup wait finished, self: 0xfcdb06ca40

……
YYYY-MM-DDTHH:MM:SS.###Z Er(155) veecdp[2111649]: [0002111824] 914 (E) [Timer] Failed to remove timer, this: 0xfcdad7d598, status 'Object not found'

YYYY-MM-DDTHH:MM:SS.###Z In(158) veecdp[2111649]: [0002111824] 914 (I) [WorkGroup] WorkGroup wait started, self: 0xfcdb06ca40
YYYY-MM-DDTHH:MM:SS.###Z In(158) veecdp[2111649]: [0002111824] 914 (I) [WorkGroup] WorkGroup wait finished, self: 0xfcdb06ca40

Environment

VMware vSphere ESXi 8.0

Cause

After upgrading ESXi to 8.0, the IO Filter was not upgraded accordingly to the Veeam CDP plug-in version compatible with 8.0 and remained at the 7.0 version. As a result, the veecdp service was in a stopped state.

Under this condition, operations such as taking snapshots, performing vMotion, or modifying storage policies on virtual machines running on ESXi 8.0 may intermittently hang.

Resolution

Engage Veeam Support to upgrade the Veeam CDP plug-in to a version that is compatible with ESXi 8.0.