Encrypted VM with Change Block Tracking (CBT) enabled unexpectedly powers off after shallow rekey operation
Article ID: 387897
Updated On:
Products
Issue/Introduction
When operating ESXi 7.x hosts connected to a vCenter 8.x and attempting to perform a shallow rekey of a powered-on, encrypted VM with CBT (Changed Block Tracking) enabled, the rekey operation will be successful, but the VM may unexpectedly power off (and subsequent HA actions conducted, if enabled).
The following error may be seen in the ESXi /var/run/log/hostd.log:
2025-01-01T00:00:00.000Z verbose hostd[######] [Originator@#### sub=Vmsvc.vm:/vmfs/volumes/<datastore>/VM.vmx opID=CdrsLoadBalancer-######-########-##-##-##-####] Handling vmx message ####: Could not open or create change tracking file
--> Cannot open the disk '/vmfs/volumes/vmfs/volumes/datastore/VM.vmdk' or one of the snapshot disks it depends on.
--> An operation required the virtual machine to quiesce and the virtual machine was unable to continue running.
Environment
vCenter 8.x with ESXi 7.x hosts
vCenter 8.x with ESXi 8.x hosts
VM is encrypted
Change Block Tracking (CBT) is enabled
Note: this issue does not occur when using ESXi 7.x hosts connected to a vCenter 7.x.
Cause
The issue occurs after a successful rekey due to a logic in a race condition on ESXi between hostd and the VMX process when attempting to access the VM's vmdk disk(s) and the corresponding ctk file(s).
Resolution
This issue will be resolved in future versions of vSphere.
The following workarounds are available:
- Power off the VM and perform the rekey operation
- Disable CBT prior to rekeying the VM, rekey the VM and then reenable CBT. (Note: an additional full VM backup should be taken after re-enabling CBT)
- Connect the hosts from a vCenter 8.x back to a vCenter 7.x and perform the operation with the ESXi 7.x host(s) connected to a vCenter 7.x
Additional Information
Feedback
