Enabling or disabling Changed Block Tracking (CBT) on virtual machines - VMware vSphere

Products

• VMware vSphere ESXi

Issue/Introduction

• Error when backing up a virtual machine:
  One or more VM disks have incorrect changed block tracking configuration.

• Error during backup job:
  Disk "Hard disk #" has incorrect changed block tracking configuration
One or more VM disks have incorrect changed block tracking configuration
  
  
• Error when consolidating snapshots:
  Unable to access file since it is locked. An error occurred while consolidating disks. One of the disks in this virtual machine is already in use by a virtual machine or by a snapshot. (In this case, the files will be there and there will be no locks present in the VM folders.)
  
  
• Error when using Storage vMotion to move a VM's storage location, which moves VMs and their associated components between Storage ( Datastore ):
  The operation is not supported on the object.
vMotion of a CBT-enabled VM having multi-writer disk(s) is not supported
  
  
• Error when attaching a disk:
  The operation is not allowed. Cannot attach a CBT enabled disk: /vmfs/volumes/########-####-####-####-############.vmdk to CBT enabled VM. 

Environment

• VMware vSphere ESXi 7.x
• VMware vSphere ESXi 8.x
• VMware vSphere ESXi 9.x

Cause

Backup products take advantage of Changed Block Tracking (CBT) for incremental backups if CBT is enabled in the virtual machine. For more information on CBT, see Changed Block Tracking (CBT) on virtual machines.

The CBT feature of VMware which facilitates taking incremental backups on virtual machines running on VMware ESXi. CBT identifies and tracks block changes since the last backup, and stores these changes as logs for tracking the backups.

Resolution

Enable CBT in a virtual machine:

1. Power off the virtual machine.
2. Right-click the virtual machine and click Edit Settings.
3. Click the Advanced Parameters tab.
4. Add the ctkEnabled parameter and then set its value to true.
5. Click Add Row, add scsi0:0.ctkEnabled, and set its value to true.
   
   Note: scsi0:0 in scsi0:0.ctkEnabled indicates the SCSI device assigned to the hard disk that is added to the virtual machine.
   Every hard disk added to the virtual machine is given a SCSI device that appears similar to scsi0:0, scsi0:1, or scsi 1:1. CBT is enabled (or disabled) individually on each disk.
    
6. Power on the virtual machine.
7. In the home directory of the virtual machine, verify that each disk having CBT enabled has also a vmname-ctk.vmdk file.

Disable CBT:

Note: It is recommended that there are no snapshots on the VM, but you can disable it even if it has snapshots. See the following KBs:

• Consolidating/Committing snapshots in VMware ESXi
• VC_FAULT_FATAL - A specified parameter was not correct: vm.ctkEnabled. Instant Clone Creation Error
• Changed Block Tracking (CBT) on virtual machines

1. Power off the virtual machine.
2. Right-click the virtual machine and click Edit Settings.
3. Click the Advanced Parameters tab..
4. Set the ctkEnabled parameter to FALSE for the ctkEnabled setting for the VM and for each SCSI disk(s).
5. Remove the .vmsd file along with any ctk.vmdk files in the VM folder(s). You can create a subfolder and move them there or just delete them.
6. Power on the virtual machine.

Note: Ensure that there are no snapshots on the virtual machine before enabling change tracking. If you create snapshots before enabling CBT, the QueryChangedDiskAreas API might not return any error or the data returned by QueryChangedDiskAreas might be incorrect.

Additional Information

You can use these PowerCLI commands to disable and then enable CBT without powering off the virtual machine. This can be useful when one or more of the disks of the virtual machine is extended past a 128Gb boundary. Example shown uses VM_Name as the $vm variable this should contain the name of the vm you are reconfiguring. 

• Set Variables

$vm="VM_Name"

$vmtest = Get-vm $vm| get-view

$vmConfigSpec = New-Object VMware.Vim.VirtualMachineConfigSpec

• Disable ctk

$vmConfigSpec.changeTrackingEnabled = $false

$vmtest.reconfigVM($vmConfigSpec)

• Enable ctk

$vmConfigSpec.changeTrackingEnabled = $true

$vmtest.reconfigVM($vmConfigSpec)

 

Note : If you enable or disable CBT using PowerCLI or the vSphere API while the virtual machine is powered ON, the configuration change is saved but CBT tracking does not immediately initialize. To complete the CBT enablement and generate the required tracking files (-ctk.vmdk), the VM must undergo a stun/unstun cycle. You must perform one of the following actions immediately after enabling CBT on a live VM:

• Create and immediately delete a virtual machine snapshot.
  Or
• Power cycle the virtual machine (Power Off, then Power On).

For steps to do a full reset of CBT data, see Resetting Changed Block Tracking for VMware vSphere virtual machines.