Prerequisites to upgrading to VKr (Previously known as TKr) 1.31 and onward for Guest Clusters with custom Antrea Resources

Products

• VMware vSphere with Tanzu
• vSphere with Tanzu

Issue/Introduction

VKr 1.31 includes Antrea version 2.1 which retires the following advanced Antrea CRD's. 

 

After upgrading the following errors will be seen in the clusterbootstrap status on the supervisor cluster. The below example is specific to the "Tier" CRD, but if other CRD's are used, they show a similar error. 

status:
conditions:

• lastTransitionTime: "2024-11-26T09:04:15Z"
message: |-
kapp: Error: update customresourcedefinition/tiers.crd.antrea.io (apiextensions.k8s.io/v1) cluster:
Updating resource customresourcedefinition/tiers.crd.antrea.io (apiextensions.k8s.io/v1) cluster:
API server says:
CustomResourceDefinition.apiextensions.k8s.io "tiers.crd.antrea.io" is invalid: status.storedVersions[0]:
Invalid value: "v1alpha1": must appear in spec.versions (reason: Invalid)

Environment

Issue can happen if any of the advanced API's were being used pre VKr 1.31, then upgraded to VKr 1.31 or higher. 

Resolution

There is an Antrea cli tool called antctl which will migrate the objects from the old CRD's to the new CRD's.

Download antctl at the bottom of this page under Assets: https://github.com/antrea-io/antrea/releases/tag/v2.1.0  

Make sure you have a .kube/config pointing to the guest cluster with a Supervisor admin role.

1. Pause the app

# kubectl patch pkgi <cluster name>-antrea -n vmware-system-tkg --type merge -p '{"spec":{"paused": true}}' 

2. Delete the webhook

# kubectl delete validatingwebhookconfigurations.admissionregistration.k8s.io crdvalidator.antrea.io
# kubectl delete mutatingwebhookconfigurations.admissionregistration.k8s.io crdmutator.antrea.io 

3. Perform the migration.

# antctl upgrade api-storage --dry-run
# antctl upgrade api-storage

4. Un-pause the app

# kubectl patch pkgi <cluster name>-antrea -n vmware-system-tkg --type merge -p '{"spec":{"paused": false}}'

5. Wait up to ten minutes max for the next KAPP resync to occur. 

6. Validate that the pkgi is healthy and showing "ReconcileSucceeded" by running 

# kubectl get pkgi <cluster name>-antrea -n vmware-system-tkg