vCenter Services unable to start after replacing Machine SSL certificate or upgrading vCenter to 8.0 U3

Products

• VMware vCenter Server

Issue/Introduction

• Attempting to upgrade to 8.0 Update 3 fails during the service restart process
• Attempting to replace the vCenter machine SSL certificate on 8.0 Update 3 fails during the service restart process

service_manager.IllegalServiceOperation: Service cannot be started. Error: Error executing start on service vpxd-svcs. Details {
    "detail": [
        {
            "id": "install.ciscommon.service.failstart",
            "translatable": "An error occurred while starting service '%(0)s'",
            "args": [
                "vpxd-svcs"
            ],
            "localized": "An error occurred while starting service 'vpxd-svcs'"
        }
    ],
    "componentKey": null,
    "problemId": null,
    "resolution": null
}

• The vmon.log (under /var/log/vmware/vmon) shows vpxd-svcs failing to start up properly during service start up

<vpxd-svcs> Service pre-start command's stderr: pyVmomi.VmomiSupport.vmodl.fault.SystemError: (vmodl.fault.SystemError) {
dynamicType = <unset>,
dynamicProperty = (vmodl.DynamicProperty) [],
msg = 'Internal server error',
faultCause = <unset>,
faultMessage = (vmodl.LocalizableMessage) [],
reason = 'javax.net.ssl.SSLException: SSL handshake from 0.0.0.0/0.0.0.0:33046 to VC_FQDN/127.0.0.1:443 failed in 103 ms'

• Checking the vpxd-svcs log under /var/log/vmware/vpxd-svcs shows complaints about the Lookup Service not being able to started successfully

2025-02-10T18:01:33.740Z ERROR pre-start-vpxd-svcs Failed to reregister Tagging service grpc endpoints with Lookup Service
2025-02-10T18:01:33.740Z ERROR pre-start-vpxd-svcs (vmodl.fault.SystemError) {
   dynamicType = <unset>,
   dynamicProperty = (vmodl.DynamicProperty) [],
   msg = 'Internal server error',
   faultCause = <unset>,
   faultMessage = (vmodl.LocalizableMessage) [],
   reason = 'javax.net.ssl.SSLException: SSL handshake from 0.0.0.0/0.0.0.0:34608 to vCENTER_FQDN/127.0.0.1:443 failed in 11 ms'

• The lookup service log (/var/log/vmware/lookupsvc/lookupserver-default.log) shows the same error seen in the vpxd-svcs log, but we see the cause pointed out in the stack trace 

INFO  com.vmware.vim.vmomi.server.impl.InvocationTask] Rejecting unauthenticated access to non-anonymous API method.
com.vmware.vim.sso.admin.exception.InternalError: javax.net.ssl.SSLException: SSL handshake from 0.0.0.0/0.0.0.0:53266 to vCENTER_FQDN/127.0.0.1:443 failed in 14 ms
...
Caused by: org.bouncycastle.tls.TlsFatalAlert: internal_error(80); Certificate chain longer than maximum (10)

Environment

vCenter Server 8.0.3

Cause

This issue occurs due to the vCenter machine SSL certificate having over 10 certificates in the certificate chain (certificate chain includes root CA, intermediate CAs and leaf certificate)

vCenter 8.0.3 does not support more than 10 certificates in the certificate chain

Resolution

To resolve this issue replace the vCenter machine SSL certificate with a new certificate which includes a certificate chain that is 10 certificates or less