TKG Service Cluster Control plane deployment Fails with Port 6443 Connectivity Errors "VirtualMachineService LB does not yet have VIP assigned, error extracting the OVF envelope"

Products

• VMware vSphere Kubernetes Service

Issue/Introduction

During the deployment of the TKG Service Cluster (formerly known as TKC), the Control Plane VIP was not reachable on port 6443.

Attempts to connect resulted in errors like:

curl -vf telnet://<TKG Service Cluster Control Plane VIP>:6443
> connection failed: Connection refused

One control plane VM came up, but the other two did not. SSH access to the running control plane VM was unsuccessful. The cluster creation process failed due to this connectivity issue.

VMOP pod in the Supervisor shows the error below.

kubectl -n vmware-system-vmop logs deployment/vmware-system-vmop-controller-manager
> failed to reconcile loadbalanced endpoint for WCPCluster: VirtualMachineService LB does not yet have VIP assigned: VirtualMachineService LoadBalancer does not have any Ingresses

Environment

VMware vSphere with Tanzu 8.x

Cause

The control plane VMs of the TKG Service Cluster were unable to communicate properly with the Supervisor.

• Network restrictions or firewall rules are blocking traffic on port 6443, preventing Kubernetes API communication
• Inter-routing issues are hindering proper communication between the Control Plane network of the TKC cluster and the Supervisor network

Resolution

TKG Service Cluster must have network connectivity with the Supervisor VIP.

• Fix the routing issue from the "Namespace Network" to the "Ingress" network
  
  • Those network is defined in Workload Network  (Workload Management --> Supervisors --> Configure --> Network)
  • Review if the firewall is blocking between those networks

After that, the TKG Service Cluster creation task will be resumed, and the cluster will be deployed successfully.