In a future TKG release, the node-pool with customized template upgrade issue will be addressedWorkaround:
Below are the steps to upgrade a legacy workload cluster with a customized template using APIs.
- Upgrade Packages
Retrieve the TKR version supported for the TKG release
tanzu kubernetes-release get
Target the Management cluster and edit the workload cluster to update metadata.labels.tanzuKubernetesRelease to target TKR version
kubectl edit cluster <workload cluster name>
apiVersion: cluster.x-k8s.io/v1beta1
kind: Cluster
metadata:
:
labels:
tanzuKubernetesRelease: v1.24.10---vmware.1-tkg.2
This will trigger the upgrade of tanzu packages if they are bumped in this version.
Target the workload cluster and check if packageRepository and packageinstalls are bumped and in ReconcileSucceeded status
kubectl config use-context <Workload Context>
kubectl get packageRepository -A
kubectl get packageinstalls -n tkg-system
- Upgrade Control Plane nodes
Target the Management cluster.
Create a new vspheremachinetemplate for the Workload Control Plane nodes
kubectl get vspheremachinetemplate
kubectl get vspheremachinetemplate <workload CP VM template> -o yaml > workload-cp.yaml
Modify metadata.name and spec.template.spec.template
vi workload-cp.yaml
apiVersion: infrastructure.cluster.x-k8s.io/v1beta1
kind: VSphereMachineTemplate
metadata:
name: work2-control-plane-1.24.10
:
spec:
template:
spec:
template: /wdc-10/vm/tkg/ubuntu-2004-kube-v1-24-10+vmware-1-tkg-1
Create the new vSphereMachineTemplate
kubectl apply workload-cp.yaml
Edit workload cluster KubeadmControlPlane to upgrade Kubernetes spec.version and change spec.machineTemplate.infrastructureRef.name reference to the new one created above.
kubectl edit kcp <workload kcp> -n <namespace>
spec
machineTemplate:
infrastructureRef:
apiVersion: infrastructure.cluster.x-k8s.io/v1beta1
kind: VSphereMachineTemplate
name: work2-control-plane-1.24.10
namespace: default
metadata: {}
replicas: 1
rolloutStrategy:
rollingUpdate:
maxSurge: 1
type: RollingUpdate
version: v1.24.10+vmware.1
This will upgrade the Control Plane nodes. The below objects can be used to track the control plane upgrade
# From Management cluster
kubectl get kcp
kuebctl get machines
# From workload cluster
kubectl get nodes
- Upgrade nodepool
For each nodepool, create a new vSphereMachineTemplate,
kubectl get vspheremachinetemplate
kubectl get vspheremachinetemplate <workload nodepool VM template> -o yaml > workload-np.yaml
Modify .metadata.name and .spec.template.spec.template
apiVersion: infrastructure.cluster.x-k8s.io/v1beta1
kind: VSphereMachineTemplate
metadata:
name: work2-worker-1.24.10
:
spec:
template:
spec:
template: /wdc-10-vc35/vm/tkg/ubuntu-2004-kube-v1-24-10+vmware-1-tkg-custom
Create the new vSphereMachineTemplate
kubectl apply -f workload-np.yaml
Update spec.template.spec.infrastructureRef.name and spec.template.spec.version in the machineDeployment
kubectl get md -A
kubectl edit md <Machine Deployment> -n <namespace>
spec:
template:
spec:
clusterName: work2
infrastructureRef:
apiVersion: infrastructure.cluster.x-k8s.io/v1beta1
kind: VSphereMachineTemplate
name: work2-worker-1.24.10
version: v1.24.10+vmware.1
This will upgrade the MachineDeployment. The below objects can be used to track the upgrade
# From Management cluster
kubectl get md
kuebctl get machines
# From workload cluster
kubectl get nodes
The "Upgrade nodepool" steps can be repeated for any additional nodepools in the cluster.
Once all nodepools have been upgraded, confirm cluster status
tanzu cluster list