Guest cluster TKr install or upgrade to 1.33 or later fails with "vmware-system-vks-public/builtin-generic-v3.1.0 set by annotations: kubernetes.vmware.com/min-version-supported and kubernetes.vmware.com/max-version-supported"
search cancel

Guest cluster TKr install or upgrade to 1.33 or later fails with "vmware-system-vks-public/builtin-generic-v3.1.0 set by annotations: kubernetes.vmware.com/min-version-supported and kubernetes.vmware.com/max-version-supported"

book

Article ID: 410526

calendar_today

Updated On:

Products

Tanzu Kubernetes Runtime

Issue/Introduction

Install or upgrade a TKC with TKR version 1.33 or later fails with below error:

Error from server (Forbidden): error when creating "<testcluster_1.33>.yaml": 
admission webhook "capi.validating.tanzukubernetescluster.run.tanzu.vmware.com" denied the request: 

Cluster's Kubernetes version v1.33.1+vmware.1-fips does not meet the version requirements of ClusterClass vmware-system-vks-public/builtin-generic-v3.1.0 set by annotations: kubernetes.vmware.com/min-version-supported and kubernetes.vmware.com/max-version-supported

We also run the command of "kubectl get tkr" and return that TKR v1.33 is True

NAME                                VERSION                          READY   COMPATIBLE   CREATED   TYPE
v1.25.7---vmware.3-fips.1-tkg.1     v1.25.7+vmware.3-fips.1-tkg.1   False  False        13d
v1.26.5---vmware.2-fips.1-tkg.1     v1.26.5+vmware.2-fips.1-tkg.1   False   False        13d
v1.27.11---vmware.1-fips.1-tkg.2    v1.27.11+vmware.1-fips.1-tkg.2   False   False        13d
v1.28.8---vmware.1-fips.1-tkg.2     v1.28.8+vmware.1-fips.1-tkg.2   False   False        13d
v1.29.4---vmware.3-fips.1-tkg.1     v1.29.4+vmware.3-fips.1-tkg.1   True    True         13d
v1.30.1---vmware.1-fips.1-tkg.5     v1.30.1+vmware.1-fips.1-tkg.5   True    True         13d
v1.31.1---vmware.2-fips-vkr.2       v1.31.1+vmware.2-fips-vkr.2      False   False        13d
v1.32.0---vmware.6-fips-vkr.2       v1.32.0+vmware.6-fips-vkr.2      True    True         13d
v1.33.1---vmware.1-fips-vkr.2       v1.33.1+vmware.1-fips-vkr.2      True    True         5d6h

We can also see the cluster classes available using the command: kubectl get clusterclass -A | grep vmware-system-vks-public

vmware-system-vks-public   builtin-generic-v3.1.0  57m
vmware-system-vks-public   builtin-generic-v3.2.0  57m
vmware-system-vks-public   builtin-generic-v3.3.0  57m
vmware-system-vks-public   builtin-generic-v3.4.0  56m
vmware-system-vks-public   tanzukubernetescluster  56m

Environment

VKR(TKR) 1.33 or later

Cause

This is due to change with VKS 3.4 release

Reference: VMware vSphere Kubernetes Service Release Notes

Deprecation Announcements:

With the release of VKS 3.4, we introduce and recommend using the builtin-generic-v3.4.0 ClusterClass. Existing clusters will need to rebase to builtin-generic-v3.4.0 as a part of, or prior to, any upgrade to Kubernetes v1.33 or later (For more information, see link) Additionally, the tanzukubernetescluster, builtin-generic-v3.1.0, builtin-generic-v3.2.0, and builtin-generic-v3.3.0 ClusterClass are considered deprecated and will be removed in future releases.

With the release of VKS 3.4, we recommend using builtin-generic-v3.4.0 as the foundation for any Custom ClusterClass. Existing Custom ClusterClass should be rebased onto builtin-generic-v3.4.0. (For more information, see link.) Custom ClusterClass based on the tanzukubernetescluster, builtin-generic-v3.1.0, builtin-generic-v3.2.0, and builtin-generic-v3.3.0 ClusterClass are also considered deprecated.

With vKR 1.33, to create or upgrade cluster we have to use Cluster v1beta1 API.

Refer: Cluster v1beta1 API

    Resolution

    TKr Clusters for 1.33 or later can only be created/upgraded with cluster class builtin-generic-v3.4.0
    To use cluster class builtin-generic-v3.4.0, edit the class (or .yaml file) as below example.

    Example:

    apiVersion: cluster.x-k8s.io/v1beta1
    kind: Cluster
    metadata:
      name: Test
      namespace: test-ns
    spec:
      clusterNetwork:
        services:
          cidrBlocks: [""]
        pods:
          cidrBlocks: [""]
        serviceDomain: "cluster.local"
      topology:
        class: builtin-generic-v3.4.0 ###>>> Here we updated the cluster class.
        version: v1.33.1+vmware.1-fips-vkr.2
        controlPlane:
          replicas: 3

    Additional Information

    Reference: Introducing VMware vSphere Kubernetes Service 3.4: Extended Kubernetes Support, Istio Service Mesh, and Enhanced Multi-Cluster Management - VMware Cloud Foundation (VCF) Blog

    VKS v3.4 introduces the recommended builtin-generic-v3.4.0 ClusterClass. Existing clusters must rebase to this Class before or during any Kubernetes v1.33+ upgrade. Older ClusterClass (tanzukubernetesclusterbuiltin-generic-v3.1.0builtin-generic-v3.2.0, and builtin-generic-v3.3.0) are deprecated and will be removed in future. (see table below).

    With VKS v3.4, use builtin-generic-v3.4.0 as the base for Custom ClusterClass, rebasing existing ones. Older Custom ClusterClass (tanzukubernetesclusterbuiltin-generic-v3.1.0builtin-generic-v3.2.0, and builtin-generic-v3.3.0) are deprecated and will be removed in future. (see table below).

    ClusterClass Version Deprecation Announcement Removal
    tanzukubernetescluster June 2025 not before October 2025
    builtin-generic-v3.1.0builtin-generic-v3.2.0, & builtin-generic-v3.3.0 June 2025 not before February 2026