vSphere Supervisor Workload Cluster creation is stuck at 'Configuring Load Balancer'

Products

• VMware vSphere Kubernetes Service

Issue/Introduction

• Deploying a Workload Cluster in a vSphere Supervisor environment, the process stalls at the "Configuring Load Balancer" stage.
• In the Supervisor Cluster context, AKO pod is found in a CrashLoopBackOff state
  
  kubectl get pods -A | grep -v Running
NAMESPACE                                   NAME                                                              READY   STATUS             RESTARTS         AGE
vmware-system-ako                           vmware-system-ako-ako-controller-manager-#########-#####          0/1     CrashLoopBackOff   13 (#m#s ago)   ##m



• AKO pod logs show a missing IPAM configuration.
  
  kubectl logs vmware-system-ako-ako-controller-manager-#########-4pd## -n vmware-system-ako



  YYYY-MM-DD:T:HH:MM:SS stdout F YYYY-MM-DD:T:HH:MM:SS         lib/avi_api.go:65       msg: Unable to fetch data from uri /api/systemconfiguration Encountered an error on GET request to URL https://<<avi-load balancer>>:443//api/systemconfiguration: HTTP code: 403; error from Avi: map[error:User '<avi-load balancer user>' is not authorized to read on resource System Configuration in tenant admin]

  YYYY-MM-DD:T:HH:MM:SS stdout F YYYY-MM-DD:T:HH:MM:SS         k8s/ako_init.go:255     Error while validating input: Cloud does not have a ipam_provider_ref configured

Environment

VMware vSphere Kubernetes Service

Cause

This issue is caused by an incorrect or missing IPAM profile within the Avi Load Balancer.

Resolution

To resolve this issue, Verify if IPAM profile is configured correctly

• Log in to the Avi (NSX Advanced Load Balancer) UI.
• Navigate to Infrastructure > Clouds.
• Edit the Cloud being used 
• Ensure a valid IPAM Profile is selected
  
  

If no profile exists, create new IPAM profile. Refer to  Configure the Avi Load Balancer Controller