Error: failed: spec.topology.variables on guest cluster deploy if adding additionalTrustedCAs using v1beta api

Products

• VMware vSphere Kubernetes Service

Issue/Introduction

Error: failed: spec.topology.variables on guest cluster deploy if adding additionalTrustedCAs  using v1beta api and a builtin-generic-v3.2.0 and higher

Customer was trying to deploy a Guest cluster on VKS using  the v1beta1 documentation 

As per the documentation, they had created a secret that includes an additional trusted CA certficate with a string name additional-ca-1 

They then tried to add the Kubernetes secret containing a CA certificate to the v1beta1 API Cluster specification:

Example from Lab:

 variables:
     - name: vmClass
       value: best-effort-large
     - name: storageClass
       value: tanzu-policy
     - name: trust
       value:
         additionalTrustedCAs:
         - name: additional-ca-1

Gives an error message like

$ kubectl apply -f guest-cluster-int-1.yaml  
Error from server (Forbidden): error when creating "guest-cluster-int-1.yaml": admission webhook "capi.mutating.tanzukubernetescluster.run.tanzu.vmware.com"
denied the request: Cluster and variable validation failed: spec.topology.variables[trust]: Invalid value: "{\"additionalTrustedCAs\":[{\"name\":\"additional-ca-1\"}]}": variable is not defined

Environment

vCenter Server 8.0 using VKS Service 3.2 and above

Cause

The specification for  trust and additionalTrustedCAs have changed and moved if you are using using builtin-generic-v3.2.0 and higher with v1beta1

Instead of json path 

spec.topology.variables[trust]

trust and additionalTrustedCAs are now under  osConfiguration with a jsonpath similar to

spec.topology.variables.[osConfiguration].[trust].[additionalTrustedCAs]

and the marker caCert  and secretRef using the name of the secret and the key from the data map field

   trust:
     additionalTrustedCAs:
       - caCert:
           secretRef:
             key: <keyname>
             name: "<keyname>"

Resolution

1. Configure cluster secret with cert and then reference cert in cluster yaml 

apiVersion: v1
data:
  additional-ca-1: <double base encoded certificate data>
kind: Secret
metadata:
  name: <trusted-ca-secret name>
  namespace: <cluster nmespace>
type: Opaque

2, If you are using builtin-generic-v3.2.0 and higher, you need to now  follow this documentation:  osConfiguration under ClusterClass Variables for Customizing a Cluster

Worked example from lab

   variables:
    - name: vmClass
      value: best-effort-small
    - name: storageClass
      value: ar-storage-profile
    - name: osConfiguration
      value:
        trust:
          additionalTrustedCAs:
            - caCert:
                secretRef:
                  key: additional-ca-1
                  name: "additional-ca-1"

Additional Information

How to Force a Cluster Configuration Update/Manual Rolling Redeployment of Nodes in a vSphere Kubernetes Cluster on ClusterClass v3.3.X