Unable to patch Default StorageClass in vSphere with Tanzu.

Products

• VMware vSphere ESXi
• VMware vSphere with Tanzu

Issue/Introduction

This KB will help to set a storageClass as default in TKC.

Symptoms:
When trying to patch a StorageClass to make it default using kubectl. This would revert back after about 60 seconds. 

# kubectl patch storageclass vwt-storage-policy -p '{"metadata": {"annotations":{"storageclass.kubernetes.io/is-default-class":"true"}}}'


# kubectl get sc -A
NAME                           PROVISIONER              RECLAIMPOLICY   VOLUMEBINDINGMODE   ALLOWVOLUMEEXPANSION   AGE
vwt-storage-policy  csi.vsphere.vmware.com   Delete          Immediate           true                   32m


# kubectl describe sc vwt-storage-policy
Name:                  vwt-storage-policy
IsDefaultClass:       No
Annotations:           storageclass.kubernetes.io/is-default-class=true
Provisioner:           csi.vsphere.vmware.com
Parameters:            svStorageClass=vwt-storage-policy
AllowVolumeExpansion:  True
MountOptions:          <none>
ReclaimPolicy:         Delete
VolumeBindingMode:     Immediate
Events:                <none>

Environment

VMware vSphere 7.0 with Tanzu

Resolution

It is recommended to specify the settings:storage:defaultClass section in original yaml configuration that defines the TKG Cluster before being initially created. 

 

 

Workaround:
To workaround this issue for already running TKC, follow below steps :

1) Login to supervisor cluster using kubectl and get the guest cluster details.

# kubectl vsphere login --server=<KUBERNETES-CONTROL-PLANE-IP-ADDRESS> --vsphere-username <VCENTER-SSO-USER>

# kubectl get tkc -A

2) Edit the TKG Cluster by running below command:

# kubectl edit tanzukubernetescluster tkg-cluster-01

Adding settings:storage:defaultClass section in the yaml configuration.

apiVersion: run.tanzu.vmware.com/v1alpha1
kind: TanzuKubernetesCluster
metadata:
  name: demo-cls
  namespace: demo
spec:
  distribution:
    fullVersion: 1.22.9+vmware.1-tkg.1.cc71bc8
  topology:
    controlPlane:
      count: 1
      class: best-effort-xsmall
      storageClass: vwt-storage-policy
    workers:
      count: 1
      class: best-effort-xsmall
      storageClass: vwt-storage-policy
  settings:
    network:
      cni:
        name: calico
      services:
        cidrBlocks: ["172.16.100.0/24"]
      pods:
        cidrBlocks: ["172.16.200.0/24"]
    storage:
      defaultClass: vwt-storage-policy

Save and exit the yaml. 

3) Login to guest cluster and monitor the storage class

# kubectl vsphere login --server=SUPERVISOR-CLUSTER-CONTROL-PLANE-IP --tanzu-kubernetes-cluster-name TANZU-KUBERNETES-CLUSTER-NAME --tanzu-kubernetes-cluster-namespace SUPERVISOR-NAMESPACE-WHERE-THE-CLUSTER-IS-DEPLOYED --vsphere-username VCENTER-SSO-USER-NAME

# kubectl get sc -A
NAME                           PROVISIONER              RECLAIMPOLICY   VOLUMEBINDINGMODE   ALLOWVOLUMEEXPANSION   AGE
vwt-storage-policy (default)   csi.vsphere.vmware.com   Delete          Immediate           true                   32m

# kubectl describe sc vwt-storage-policy
Name:                  vwt-storage-policy
IsDefaultClass:        Yes
Annotations:           storageclass.kubernetes.io/is-default-class=true
Provisioner:           csi.vsphere.vmware.com
Parameters:            svStorageClass=vwt-storage-policy
AllowVolumeExpansion:  True
MountOptions:          <none>
ReclaimPolicy:         Delete
VolumeBindingMode:     Immediate
Events:                <none>

Additional Information

Create Storage Policies for vSphere with Tanzu

https://docs.vmware.com/en/VMware-vSphere/7.0/vmware-vsphere-with-tanzu/GUID-544286A2-A403-4CA5-9C73-8EFF261545E7.html