wcp service failed to start automatically with error "Failed to load host nodes from db"

Products

• VMware vCenter Server 8.0

Issue/Introduction

• wcpsvc service may not start automatically but a manual restart works.
• This happens after updating vCenter server 8.0U3a or above.
• In the wcpsvc.log below entries are seen

yyyy-mm-ddThh:mm:ss debug wcp [kubelifecycle/spherelet_lifecycle.go:268] Found cluster with clusterID: domain-<number>
yyyy-mm-ddThh:mm:ss  warning wcp [kubelifecycle/kube_instance.go:2058] [opID=domain-<id>] Failed to check if cpvm resize is supported for cluster domain-<number>. Err API call to VMware vCenter Server (vpxd) failed. Details 'object references is empty'
yyyy-mm-ddThh:mm:ss info wcp [kubelifecycle/kube_instance.go:3858] [opID=domain-<number>] Adding Host host-<id>to supervisor <id>
yyyy-mm-ddThh:mm:ss error wcp [kubelifecycle/kube_instance_db.go:889] [opID=domain-<number>] Failed to load host nodes from db: failed to add host to supervisor: host host-<id> not found in inventory
yyyy-mm-ddThh:mm:ss  fatal wcp [kubelifecycle/kube_instance.go:1105] Failed to load kubeInstances from db: failed to add host to supervisor: host host-<id> not found in inventory
yyyy-mm-ddThh:mm:ss debug wcp [compatibility/setupinfo_host_monitor.go:86] [opID=HostCacheMonitor] Initializing host cache monitor.
yyyy-mm-ddThh:mm:ss  debug wcp [compatibility/setupinfo_vpxd_cluster_monitor.go:70] [opID=ClusterMonitor] Initializing cluster monitor.
...
yyyy-mm-ddThh:mm:ss debug wcp [compatibility/setupinfo.go:1202] [opID=upsertHostCluster] Added host host-<id> to cluster domain-<number>. ESX version:  0 pkgs 0 cores

Environment

VMware vCenter Server 8.0 U3a
VMware vCenter Server 8.0 U3b
VMware vCenter Server 8.0 U3c
VMware vCenter Server 8.0 U3d

Cause

This happens due to race condition in host sync during wcp service startup

Resolution

This is a known issue and currently there is no resolution.

Workaround

Manually start "Workload Control Plane" (wcp) service from the vCenter Server Management Interface (VAMI) UI or by going to vCenter Server shell and running command:

service-control --start wcp

If the problem occurs during patching of vCenter, follow the same procedure as above once it fails, and then click the Resume vCenter update option and select "Apply" (See screenshot):