Supervisor enablement stuck on Foundation Load Balancer (FLB) deployment. Error "Timed out waiting for LB service update"

Products

• VMware vSphere Kubernetes Service
• VMware vCenter Server

Issue/Introduction

• When deploying supervisor with Foundation Load Balancer (FLB) could get stuck at FLB deployment stage.
  
  
• Supervisor control plane VMs were deployed successfully but the FLB VMs are not present in the inventory.
  
  
• The Supervisor Management page shows the Supervisor cluster in Configuring state and will time-out with the error:
  

  Configured Load Balancer fronting the kubernetes API Server
  Timed out waiting for LB service update. This operation is part of the cluster enablement and will be retried

  


• On vCenter VM, the following logs indicate Solution task failures for FLB and reference the domain ID for the deploying Supervisor cluster:
  

  /var/log/vmware/vpxd/vpxd.log
  
  YYYY-MM-DDTHH:MM:SS error vpxd[#####] [Originator@1234 sub=FoundationLoadBalancersSvc opID=FLB-Monitor-CheckCompliance-######] vLCM API invocation failed; cluster: domain-c####, method: SetSolution, taskId: ######-####-####-####-#######:com.vmware.esx.settings.clusters.vms.solutions, error: Task failed with Unknown Error: Unknown Exception
  YYYY-MM-DDTHH:MM:SS error vpxd[#####] [Originator@1234 sub=FoundationLoadBalancersSvc opID=FLB-Monitor-CheckCompliance-######] Failed to set solution; cluster id: domain-c####, solution id: foundation-loadbalancer-1, info: {
  
  /var/log/vmware/vmware-updatemgr/vum-server/vmware-vum-server.log
  
  YYYY-MM-DDTHH:MM:SS info vmware-vum-server[11070] [Originator@1234 sub=com.vmware.vcIntegrity.lifecycle.lccm.SetSolutionTask] [SolutionsSvcSetTask, 71] Task:com.vmware.vcIntegrity.lifecycle.lccm.SetSolutionTask ID:######-####-####-####-#######. Storing solution "foundation-loadbalancer-1" for cluster "domain-c####"
  YYYY-MM-DDTHH:MM:SS error vmware-vum-server[11070] [Originator@1234 sub=Default] [VdbStatement] Execute result code: -1
  YYYY-MM-DDTHH:MM:SS error vmware-vum-server[11070] [Originator@1234 sub=Default] [VdbStatement] SQL execution failed: INSERT INTO SYSTEM_VMS_CLUSTER_SOLUTION_SPEC_VM_PLACEMENT_POLICIES (CLUSTER_ID, SOLUTION_ID, VM_PLACEMENT_POLICY) VALUES (?, ?, ?)
  YYYY-MM-DDTHH:MM:SS error vmware-vum-server[11070] [Originator@1234 sub=Default] [VdbStatement] Execution elapsed time: 1 ms
  YYYY-MM-DDTHH:MM:SS error vmware-vum-server[11070] [Originator@1234 sub=Default] [VdbStatement] Statement diagnostic data from driver is 42703:0:1:ERROR: column "vm_placement_policy" of relation "system_vms_cluster_solution_spec_vm_placement_policies" does not exist;
  --> Error while preparing parameters

Environment

vCenter Server 9.x

Cause

The execution of the vum-db-reset (/usr/lib/vmware-updatemgr/bin/updatemgr-utility.py reset-db) operation on vCenter Server removed essential database entries required for lifecycle operations.

Resolution

Open a support request with VMware by Broadcom Support referencing this KB article for assistance with fixing the vCenter Server database table entries.

Additional Information

Reset VMware Lifecycle Manager depot in vCenter Server Appliance

Creating and managing Broadcom support cases