Kubernetes and OS version drop-down fields do not have any values when creating clusters in TMC

Products

• VMware Tanzu Mission Control
• VMware vSphere with Tanzu

Issue/Introduction

When trying to create a new cluster in TMC using a "vSphere with Tanzu" Supervisor Cluster as the Provisioner, the "Kubernetes version" and "OS version" drop-down fields do not have any values, hence the cluster creation form cannot be completed.

Environment

VMware Tanzu Mission Control

VMware vSphere with Tanzu

Cause

The TMC pod "vsphere-resource-retriever" could be stuck in a loop and failing to retrieve information from the Supervisor cluster.  This pod is in the "svc-tmc-cN" namespace and could be showing as "Running" even if the issue is happening.

$ kubectl get pods -n svc-tmc-c7
NAME                                                              READY   STATUS      RESTARTS       AGE
agent-updater-XXcf84d99-yyyyy                                     1/1     Running     0              18d
agentupdater-workload-XX936530-yyyyy                              0/1     Completed   0              17s
cluster-health-extension-XX4d8c7b-yyyyy                           1/1     Running     0              18d
extension-manager-XXfdbbc5c-yyyyy                                 1/1     Running     1 (10d ago)    18d
extension-updater-XX5d4789f4-yyyyy                                1/1     Running     1 (1d1h ago)   18d
intent-agent-XXf4dfd7c5-yyyyy                                     1/1     Running     0              18d
sync-agent-XX6cb694c9-yyyyy                                       1/1     Running     0              18d
tmc-agent-installer-XX936530-yyyyy                                0/1     Completed   0              17s
tmc-auto-attach-XX4f646fcd-yyyyy                                  1/1     Running     0              18d
vsphere-resource-retriever-XXXXYYZZZZ-yyyyy                       1/1     Running     0              18d

The logs of the pod could have this repeating error:

E0107 17:43:49.443029       1 reflector.go:140] pkg/mod/k8s.io/client-go@v0.25.4/tools/cache/reflector.go:169: Failed to watch *v1alpha1.VirtualMachineClassBinding: failed to list *v1alpha1.VirtualMachineClassBinding: the server could not find the requested resource (get virtualmachineclassbindings.vmoperator.vmware.com)

Resolution

As a workaround, delete the "vsphere-resource-retriever" pod to effectively restart it.

kubectl -n svc-tmc-c7 delete pod vsphere-resource-retriever-XXXXYYZZZZ-yyyyy

Once the pod is back up and running and there are no more errors in its logs, try to create the cluster again from TMC.