Supervisor deployment fails with error "Timed out waiting for LB service update. This operation is part of the cluster enablement and will be retried"

Products

• VMware vSphere Kubernetes Service
• VMware vCenter Server

Issue/Introduction

• Supervisor deployment fails with the below error in an NSX and Avi Load Balancer environment.
  
  


• The error occurs even when using distributed switch, NSX Load Balancer or Avi Load Balancer for the supervisor configuration.
• The below errors can be seen in the vCenter's /var/log/vmware/wcp/wcpsvc.log:
  
  YYYY-MM-SSThh:mm:ssZ error wcp [apiserver/manager.go:###] [opID=######-######-######] timed out waiting for kube-apiserver-lb-svc service update. Err: context deadline exceeded
YYYY-MM-SSThh:mm:ssZ error wcp [kubelifecycle/controller.go:###] [opID=######-######-######] An error occurred fetching the virtual IP: Timed out waiting for LB service update. This operation is part of the cluster enablement and will be retried.. Retrying
YYYY-MM-SSThh:mm:ssZ debug wcp [kubelifecycle/controller.go:###] [opID=######-######-######] Supervisor configuration retry.

Environment

• vCenter server 8.x
• NSX-T
• Avi Load Balancer

Cause

When NSX Avi Load Balancer is integrated with NSX Manager, the Supervisor enablement wizard prioritizes Avi as the default Load Balancer provider. This issue occurs if the Avi Load Balance is in degraded state (e.g., due to Service Engine failures or configuration or synchronization issues).

Resolution

To resolve this issue, the load-balancing infrastructure must be brought back to a healthy state before re-attempting the Supervisor deployment:

1. Log in to the Avi Load Balancer and NSX Manager UI and identify what is reporting a "Degraded" or "Down" status.
2. Contact your internal Avi/Load Balancing administration team or Broadcom support to investigate and remediate the underlying issue.
3. Once the Avi Load balancer issue is resolved. Return to Workload Management page in vCenter to monitor the Supervisor deployment status.

Additional Information

Note: If you intend to use the native NSX-T Load Balancer, the Avi registration must be removed from the NSX Manager before initiating the supervisor deployment.