How to Enable CBT on worker nodes

Products

• VMware vSphere with Tanzu

Issue/Introduction

Change Block Tracking (CBT) is a VMkernel feature that keeps track of the storage blocks of a virtual machine as they change over time. The VMkernel keeps track of block changes on virtual machines, which enhances the backup process for applications that have been developed to take advantage of VMware's vStorage APIs.

If CTB is not enabled in a worker node and enabled on First Class Disks (FCDs), you can have errors on attaching PVC: PVCs cannot be mounted - The operation is not allowed in the current state.

Cause

CBT is enabled on the PVCs/FCDs but is not enabled on the newly deployed worker virtual machines. Those VMs where having problems attaching PVCs: PVCs cannot be mounted - The operation is not allowed in the current state

This is an expected behavior when the CBT flags on the FCD and the VM are different.

Resolution

To resolve this issue, we need to enable CBT on the worker VM.

1. SSH to vCenter as root
   
   
2. Login into Supervisor cluster node by using credentials from /usr/lib/vmware-wcp/decryptK8Pwd.py. Refer Troubleshooting vSphere with Tanzu (TKGS) Supervisor Control Plane VMs
   
   
3. Check the current context
   kubectl config get-contexts
   
   context should be 'kubernetes-admin'
   
   
4. Enable CBT for only one Tanzu Kubernetes Cluster Node (VM)
   kubectl edit virtualmachine <VM-Name> -n <Your-vSphere-Namespace (e.g.: shared-services)>
   
   Add following lines under Spec:
   
   advancedOptions:
    changeBlockTracking: true

Check if CBT is enabled, run this command:

kubectl get vm <vmname> -n <namespace> -o jsonpath='{.status.changeBlockTracking}'