Configure Control Plane extraArgs in workload clusters

Products

VMware Tanzu Kubernetes Grid Management VMware Tanzu Kubernetes Grid Plus VMware Tanzu Kubernetes Grid Plus 1.x

Issue/Introduction

This KB outlines how to make changes in the Control Plane components' extra arguments for TKGm workload clusters.
The procedure will depend on whether the clusters are of legacy (plan-based) or classy (ClusterClass-based) type.

A list of available arguments for each Control Plane component can be found in:

Resolution

Classy (ClusterClass-based) clusters:

New cluster creation:

For clusters created with a configuration file, you can include extra arguments in the config file itself.
Refer to the "EXTRA_ARGS" variables in the following Docs: Kubernetes and Package Tuning (Class-based clusters only)
For clusters created with a Cluster object YAML file, you can include the extraArgs in the KubeadmControlPlane section of your file, in ".spec.kubeadmConfigSpec.clusterConfiguration.<component-name>".

E.g.: following example includes "bind-address: 0.0.0.0" for the kube-controller-manager and kube-scheduler pods and "listen-metrics-urls: "http://0.0.0.0:2381"" for etcd.

spec:
kubeadmConfigSpec:
clusterConfiguration:
controllerManager:
extraArgs:
cloud-provider: external
tls-cipher-suites: TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384
bind-address: 0.0.0.0
etcd:
local:
dataDir: /var/lib/etcd
extraArgs:
cipher-suites: TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384
experimental-initial-corrupt-check: "true"
listen-metrics-urls: "http://0.0.0.0:2381"
imageRepository: projects.registry.vmware.com/tkg
imageTag: v3.5.6_vmware.14
scheduler:
extraArgs:
tls-cipher-suites: TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384
bind-address: 0.0.0.0

Existing clusters:

For existing clusters, you can edit the KubeadmControlPlane object including the necessary changes in ".spec.kubeadmConfigSpec.clusterConfiguration.<component-name>".

E.g.:

# kubectl edit kcp <kcp-object-name> -n <namespace>

This will trigger a rollout of new Control Plane nodes with the new extraArgs.

Legacy (plan-based) clusters:

New cluster creation:

The recommended way to configure legacy clusters with variables that are not settable by default in the configuration file used to create the clusters, is through ytt overlays:
Legacy Cluster Configuration with ytt

To add extraArgs in ".spec.kubeadmConfigSpec.clusterConfiguration.<component-name>" you can do as follows:

Create a ".config/tanzu/tkg/providers/infrastructure-vsphere/ytt/extraargs-overlay-test.yaml" file with the target configuration.

E.g.:

#@ load("@ytt:overlay", "overlay")
#@ load("@ytt:data", "data")
#@overlay/match by=overlay.subset({"kind":"KubeadmControlPlane"})
---
spec:
kubeadmConfigSpec:
clusterConfiguration:
controllerManager:
extraArgs:
#@overlay/match missing_ok=True
bind-address: 0.0.0.0
scheduler:
extraArgs:
#@overlay/match missing_ok=True
bind-address: 0.0.0.0
etcd:
local:
extraArgs:
#@overlay/match missing_ok=True
listen-metrics-urls: "http://0.0.0.0:2381"

Please, make sure you don't include any "tab" in the indentation, as ytt only accepts single spaces.
Trigger the cluster creation using a regular config file:

E.g.:

# tanzu cluster create -f .config/tanzu/tkg/clusterconfigs/<config-file>.yaml

If you see ytt errors such as "Expected number of matched nodes to be 1, but was 0", review the overlay file and look for "tab spaces" and errors in the indentation.
The cluster will be created including the extraArgs from the above overlay.

Existing clusters:

For existing clusters, you can edit the KubeadmControlPlane object including the necessary changes in ".spec.kubeadmConfigSpec.clusterConfiguration.<component-name>".

E.g.:

# kubectl edit kcp <kcp-object-name> -n <namespace>

This will trigger a rollout of new Control Plane nodes with the new extraArgs.