Supervisor Cluster Deployment Stalls During "Configured Core Supervisor Services"

Products

• VMware vSphere Kubernetes Service

Issue/Introduction

• During the deployment of a new vSphere with Tanzu or TKGs deployment, the cluster stalls at Configured Core Supervisor Services with the below messages:
  
  Service: tkg.vspherevmware.com. Reason: ReconcileFailed. Message: Expected to find at least one version, but did not (details:all=1 -> after-prereleases-filter=1 -> after-kubernetes-version-check=1 -> after-kapp-controller-version-check=1 -> after-constraints-filter=0).

Service: velero.vspherevmware.com. Reason: ReconcileFailed. Message: Expected to find at least one version, but did not (details:all=0 -> after-prereleases-filter=0 -> after-kubernetes-version-check=0 -> after-kapp-controller-version-check=0 -> after-constraints-filter=0).
  
  
  
  
• Following log entries are seen on the vCenter appliance under /var/log/vmware/wcp/wcpsvc.log:
  
  YYYY-MM-DDTHH:MM:SSZ error wcp [vclib/guestop_manager.go:54] [opID=<op_id>] Guest operation failed with err ServerFaultCode: File /etc/vmware/wcp/.node_configuration_status was not found
YYYY-MM-DDTHH:MM:SSZ info wcp [vclib/guestop_manager.go:57] [opID=<op_id>] Guest operation failure is not retriable.
YYYY-MM-DDTHH:MM:SSZ  debug wcp [kubelifecycle/master_node_bootstrap.go:189] [opID=<op_id>] Supervisor node bootstrap configure operation is still in progress.
YYYY-MM-DDTHH:MM:SSZ error wcp [vclib/guestop_manager.go:54] [opID=<op_id>] Guest operation failed with err ServerFaultCode: File /etc/vmware/wcp/.node_configuration_status was not found

Environment

vSphere Kubernetes Service
VMware Tanzu Supervisor Service: v3.0.0

Resolution

Restart WCP service on the vCenter Server:

service-control --stop wcp && service-control --start wcp

Additional Information

Refer following KBs for similar issues:

• Post upgrade, Supervisor stuck in "Configuring" because it cannot configure the Core Supervisor Services.
• vSphere Supervisor Disk Space Clean Up Scripts