Cannot hot add CBRC (digest) disks to non-CBRC VMs in vSphere 8.0 Update 3, "CBRC cannot be hot-enabled on a disk..."
search cancel

Cannot hot add CBRC (digest) disks to non-CBRC VMs in vSphere 8.0 Update 3, "CBRC cannot be hot-enabled on a disk..."

book

Article ID: 397165

calendar_today

Updated On:

Products

VMware vSphere ESXi

Issue/Introduction

  • CBRC (Content-based Read Cache) is enabled for Horizon virtual machines, by either
    • Enabling the option "Use View Storage Accelerator, or
    • Explicitly enabling CBRC for the hosts the VMs are running on,
    • Or both
  • However, when attempting to create snapshot-based backups of these VMs, a hot-add error is being triggered for the backup VM/backup proxy, stating: 
    CBRC cannot be hot-enabled on a disk unless the VM was powered-on with digest disks or cbrc.cache Enable was set
  • The backup software shows similar errors, for example in Veeam Backup and Replication: 
    Unable to hot add source disk <disk_name>.vmdk, failing over to network mode
  • When reviewing the host daemon log, /var/run/log/hostd.log, the following entries can be found: 
    <timestamps> In(166) Hostd[2102278]: [Originator@6876 sub=Vmsvc.vm:/vmfs/volumes/<datastore>/<backup_proxy_or_app_VM_folder>/<backup_proxy_or_app_VM_name>.vmx opID=#### sid=### user=vpxuser:<domain>/<user>] Reconfigure failed: N3Vim5Fault20GenericVmConfigFault9ExceptionE(Fault cause: vim.fault.GenericVmConfigFault
<timestamps> In(166) Hostd[2102236]: --> )
<timestamps> In(166) Hostd[2102236]: --> [context]zKq7AVICAgAAAKckdwEWaG9zdGQAAOPJR2xpYnZtYWNvcmUuc28AgU+BHQFsaWJ2aW0tdHlwZXMuc28AAWF26IKpShEBaG9zdGQAgt4iDAGCHDUMAYKlPgwBAqdU/AKNffwCx4b8AqyN/AIKc/gCOkP5Ar9R+QJlVPmBMAoWAQK1lWIAHtssAOD/LAA7UFIDUngAbGlicHRocmVhZC5zby4wAAQ/Ug9saWJjLnNvLjYA[/context]
<timestamps> Wa(164) Hostd[2102278]: [Originator@6876 sub=Vmsvc.vm:/vmfs/volumes/<datastore>/<backup_proxy_or_app_VM_folder>/<backup_proxy_or_app_VM_name>.vmx opID=#### sid=### user=vpxuser:<domain>/<user>] Exception thrown during reconfigure: (vim.vm.ConfigSpec) {
<timestamps> Wa(164) Hostd[2102236]: -->    createDate = "<timestamp>",
<timestamps> Wa(164) Hostd[2102236]: -->    files = (vim.vm.FileInfo) {
<timestamps> Wa(164) Hostd[2102236]: -->       vmPathName = "[]/vmfs/volumes/<datastore>/<backup_proxy_or_app_VM_folder>/<backup_proxy_or_app_VM_name>.vmx",
<timestamps> Wa(164) Hostd[2102236]: -->    },
<timestamps> Wa(164) Hostd[2102236]: -->    deviceChange = (vim.vm.device.VirtualDeviceSpec) [
<timestamps> Wa(164) Hostd[2102236]: -->       (vim.vm.device.VirtualDeviceSpec) {
<timestamps> Wa(164) Hostd[2102236]: -->          operation = "add",
<timestamps> Wa(164) Hostd[2102236]: -->          device = (vim.vm.device.VirtualDisk) {
<timestamps> Wa(164) Hostd[2102236]: -->             key = 0,
<timestamps> Wa(164) Hostd[2102236]: -->             backing = (vim.vm.device.VirtualDisk.FlatVer2BackingInfo) {
<timestamps> Wa(164) Hostd[2102236]: -->                fileName = "[]/vmfs/volumes/<datastore>/<backed_up_VM>/<VM_name>.vmdk",
<timestamps> Wa(164) Hostd[2102236]: -->                diskMode = "independent_nonpersistent",
<timestamps> Wa(164) Hostd[2102236]: -->             },
<timestamps> Wa(164) Hostd[2102236]: -->             connectable = (vim.vm.device.VirtualDevice.ConnectInfo) {
<timestamps> Wa(164) Hostd[2102236]: -->                startConnected = true,
<timestamps> Wa(164) Hostd[2102236]: -->                allowGuestControl = false,
<timestamps> Wa(164) Hostd[2102236]: -->                connected = true,
<timestamps> Wa(164) Hostd[2102236]: -->             },
<timestamps> Wa(164) Hostd[2102236]: -->             controllerKey = 1000,
<timestamps> Wa(164) Hostd[2102236]: -->             unitNumber = 2,
<timestamps> Wa(164) Hostd[2102236]: -->             capacityInKB = 0,
<timestamps> Wa(164) Hostd[2102236]: -->          },
<timestamps> Wa(164) Hostd[2102236]: -->       }
<timestamps> Wa(164) Hostd[2102236]: -->    ],
<timestamps> Wa(164) Hostd[2102236]: --> }
<timestamps> Wa(164) Hostd[2102236]: --> N3Vim5Fault20GenericVmConfigFault9ExceptionE(Fault cause: vim.fault.GenericVmConfigFault
<timestamps> Wa(164) Hostd[2102236]: --> )
<timestamps> Wa(164) Hostd[2102236]: --> [context]zKq7AVICAgAAAKckdwEWaG9zdGQAAOPJR2xpYnZtYWNvcmUuc28AgU+BHQFsaWJ2aW0tdHlwZXMuc28AAWF26IKpShEBaG9zdGQAgt4iDAGCHDUMAYKlPgwBAqdU/AKNffwCx4b8AqyN/AIKc/gCOkP5Ar9R+QJlVPmBMAoWAQK1lWIAHtssAOD/LAA7UFIDUngAbGlicHRocmVhZC5zby4wAAQ/Ug9saWJjLnNvLjYA[/context]

Environment

VMware vSphere ESXi 8.0.3

Cause

vSphere 8.0 Update 3 has introduced a change to the CBRC behavior.

To reduce memory consumption for the CBRC filter, from this version on hot-adding CBRC-enabled (digest) disks will only be allowed if the target VM already has a digest disk attached, or if CBRC is explicitly enabled for the VM.

Resolution

To solve this problem, ensure that cbrc.cacheEnable = "TRUE" is set on both the virtual machine being backed up and the backup server VM.

Apply the following steps:

  • Shut down the virtual machines.

  • Connect to the ESXi host where the VMs are located via SSH and log in as the root user.

  • Navigate to the virtual machine folder (typically /vmfs/volumes/<datastore>/<VM_name>/).

  • Create a backup copy of the configuration file(.vmx) by running the following command: cp <vm-name>.vmx <vm-name>.vmx.bak

  • Edit the .vmx configuration file using the VI editor: vi <vm-name>.vmx

  • Enable Content-Based Read Cache (CBRC) by adding the following line to the end of the file: cbrc.cacheEnable = "TRUE"

  • Save the file and exit the editor.

  • Reload the vmx, please see KB article: Reloading a vmx file without removing the virtual machine from inventory

  • Power the VM back on and test the backup. 

Powered by Wolken Software