TKGm pod tanzu-addon-controller-manager in CrashLoopBackOff - no matches for kind "Cluster"
book
Article ID: 313130
calendar_today
Updated On:
Products
VMware Tanzu Kubernetes Grid Management
Issue/Introduction
Symptoms: TKGm Management cluster creation fails and tanzu-addon-controller-manager pod is in CrashLoopBackOff state. Logs from the pod show that the "Cluster" CRD has not been created on the cluster.
controller-runtime/source "msg"="if kind is a CRD, it should be installed before calling Start" "error"="no matches for kind \"Cluster\" in version \"cluster.x-k8s.io/v1alpha3\""
Presence of the Cluster API CRD can be verified with: kubectl api-resources | grep cluster.x-k8s.io
The tkr-controller-manager pod downloads the BOM files from the from the VMware registry or private registry for air-gapped environment. These BOM files contain the Cluster API CRDs.
Review the tkr-controller-manager pod logs and check for the presence of the error below:
{"error": "failed to sync up TKRs with the BOM repository: failed to reconcile the BOM ConfigMap: failed to list current available BOM image tags: Get \"https://REGISTRY-FQDN": dial tcp REGISTRY-IP:443: i/o timeout"}
Resolution
Verify connectivity to the registry kubectl get pods -n tkr-system -o wide kubectl get nodes -o wide ssh capv@NODE-IP curl -kv https://REGISTRY-FQDN:443
Verify that you can download images from the registry crictl pull REGISTRY-FQDN/IMAGE-PATH
If the above verification steps are successful, check for the known issue described in https://kb.vmware.com/s/article/86496.
Apply the workaround in this KB to the Worker VM and restart the tkr-controller-manager pod.
To apply the workaround at cluster creation time, copy the contents below in a file and save in ~/.tanzu/tkg/providers/infrastructure-vsphere/ytt. See documentation for further details on overlays as the directory is different for different TKG versions.
#@ load("@ytt:overlay", "overlay")
#@overlay/match by=overlay.subset({"kind":"KubeadmControlPlane"})
---
spec:
kubeadmConfigSpec:
preKubeadmCommands:
#@overlay/append
- ethtool -K eth0 tx-udp_tnl-segmentation off && ethtool -K eth0 tx-udp_tnl-csum-segmentation off && ethtool -K eth0 tx off && ethtool -K eth0 rx off
#@overlay/match by=overlay.subset({"kind": "KubeadmConfigTemplate"})
---
spec:
template:
spec:
preKubeadmCommands:
#@overlay/append
- ethtool -K eth0 tx-udp_tnl-segmentation off && ethtool -K eth0 tx-udp_tnl-csum-segmentation off && ethtool -K eth0 tx off && ethtool -K eth0 rx off