How to Enable CBT on worker nodes

Products

• VMware vSphere with Tanzu

Issue/Introduction

Change Block Tracking (CBT), is a VMkernel feature that keeps track of the stoirage blocks of a virtual machines they change over time. The VMkernel keeps track of block .changes on virtual machines, witch enhaces the backup process for applications that h ave been developed to take advantage of VMware's vStorage APIs

 

If CTB is not enabled in a worker node and enabled on First Class Disks (FCDs), you can have errors on attaching PVC: PVCs cannot be mounted - The operation is not allowed in the current state.

Cause

CBT is enabled on the PVCs/FCDs but was not enabled on the newly deployed worker virtual machines. Those VMs where having problems attaching PVCs: PVCs cannot be mounted - The operation is not allowed in the current state

This is an expected behavior when the CBT flags on the FCD and the VM are different.

Resolution

To resolve this issue you need to enable CBT on the worker VM.

1. SSH to vCenter as root
   
   
2. login into Supervisor cluster node by using credentials from /usr/lib/vmware-wcp/decryptK8Pwd.py
   
   
3. Check the current context
   kubectl config get-contexts
   
   context should be 'kubernetes-admin'
   
   
4. enable CBT for only one Tanzu Kubernetes Cluster Node (VM)
   kubectl edit virtualmachine <VM-Name> -n <Your-vSphere-Namespace (e.g.: shared-services>)
   
   Add following lines under Spec:
   
   advancedOptions:
    changeBlockTracking: true

 

Then check if CBT is enabled, run this command:

kubectl get vm <vmname> -n <namespace> -o jsonpath='{.status.changeBlockTracking}'

Additional Information

Impact/Risks:
Please consult with your backup vendor for any possible impacts if you decide to disable Changed Block Tracking on the Virtual Machines as part of the resolution below..