vCenter Server services vmware-sps and vpxd-svsc fail to start on vCenter Server reboot - manual service starts fails also

Products

• VMware vCenter Server

Issue/Introduction

• Trying to manually start all services using command service-control --start --all results in an error -

Service-control failed. Error: Failed to start services in profile ALL. RC-1, stderr-Failed to start sps, vpxd-sves services. Error: Operation timed out.

• The error below is observed in /var/log/vmware/vpxd-svcs/pre-start-vpxd-svcs.log - 

YYYY-MM-DDThh:mm:ss.sssZ ERROR
pre-start-vpxd-svcs Failed to reregister Tagging service grpc endpoints with Lookup Service
YYYY-MM-DDThh:mm:ss.sssZ ERROR pre-start-vpxd-avcs (vmodl. fault. InvalidArgument) (
dynamicType - <unset>,
dynamicProperty = (vmodl.DynamicProperty) [],

faultCause = <unget>,
faultMessage - (vmodl. LocalizableMessage) [],
invalidProperty = 'Invalid certificate'

• Using the vCert utility - https://knowledge.broadcom.com/external/article?articleNumber=385107 - by running the commands below in it, it is observed that an incorrect Endpoint Certificate is being used as an SSL Trust Anchor

4. Manage SSL trust anchors

5. Check SSL Trust Anchors

 

Environment

vCenter Server 8.x

Cause

One or more service registrations in lookupservice are using a different certificate as their trust anchor other than the vCenter's machine SSL certificate.

Resolution

• Using the vCert utility to correct the SSL Trust Anchor anomaly -

4. Manage SSL trust anchors

5. Update SSL Trust Anchors

• Restart all vCenter Services

service-control --stop --all && service-control --start --all

• The issue is then expected to have been resolved