Certificate replacement on vCenter Server fails at 85% while starting services with error: path length constraint exceeded

Products

• VMware vCenter Server

Issue/Introduction

• Replacing custom certificate on vCenter Server fails while starting services if there are multiple CA certificate in the chain
• /var/log/vmware/vmcad/certificate-manager.log:

Service-control failed. Error: Failed to start services in profile ALL. RC=1, stderr=Failed to start sps, updatemgr, vsphere-ui, vstats, hvc, topologysvc, certificatemanagement, vpxd-svcs services. Error: Operation timed out

YYYY-MM-DDTHH:MM:SS.125Z ERROR certificate-manager  None
YYYY-MM-DDTHH:MM:SS.125Z ERROR certificate-manager  Error while starting services, please see service-control log for more details
YYYY-MM-DDTHH:MM:SS.125Z ERROR certificate-manager  Error while performing Cert Replacement operation, please see /var/log/vmware/vmcad/certificate-manager.log for more information.
YYYY-MM-DDTHH:MM:SS.125Z ERROR certificate-manager  {
    "detail": [
        {
            "id": "install.ciscommon.command.errinvoke",
            "translatable": "An error occurred while invoking external command : '%(0)s'",
            "args": [
                "None"
            ],
            "localized": "An error occurred while invoking external command : 'None'"
        },
        "Error while starting services, please see service-control log for more details"
    ],
    "componentKey": null,
    "problemId": null,
    "resolution": null
}
YYYY-MM-DDTHH:MM:SS.126Z INFO certificate-manager  Performing rollback of Root Cert...

• /var/log/vmware/vmon/vmon.log:

    YYYY-MM-DDTHH:MM:SS.475Z Wa(03) host-2258494 <vpxd-svcs> Service pre-start command's stderr: Traceback (most recent call last):
    YYYY-MM-DDTHH:MM:SS.475Z Wa(03)+ host-2258494   File "/usr/lib/vmware-vpxd-svcs/scripts/linux/pre-start/main.py", line 202, in <module>
    YYYY-MM-DDTHH:MM:SS.475Z Wa(03)+ host-2258494     endpoint_registration_runner()
    YYYY-MM-DDTHH:MM:SS.475Z Wa(03)+ host-2258494   File "/usr/lib/vmware-vpxd-svcs/scripts/linux/pre-start/main.py", line 90, in endpoint_registration_runner
    YYYY-MM-DDTHH:MM:SS.475Z Wa(03)+ host-2258494
    YYYY-MM-DDTHH:MM:SS.475Z Wa(03) host-2258494 <vpxd-svcs> Service pre-start command's stderr:     UpdateTaggingServiceGrpcEndpoint(logger).run()
    YYYY-MM-DDTHH:MM:SS.475Z Wa(03)+ host-2258494   File "/usr/lib/vmware-vpxd-svcs/scripts/linux/pre-start/tagging_grpc_registration.py", line 51, in run
    YYYY-MM-DDTHH:MM:SS.475Z Wa(03)+ host-2258494     self.update_endpoints()
    YYYY-MM-DDTHH:MM:SS.475Z Wa(03)+ host-2258494   File "/usr/lib/vmware-vpxd-svcs/scripts/linux/pre-start/tagging_grpc_registration.py", line 80, in update_endpoints
    YYYY-MM-DDTHH:MM:SS.475Z Wa(03)+ host-2258494
    YYYY-MM-DDTHH:MM:SS.475Z Wa(03) host-2258494 <vpxd-svcs> Service pre-start command's stderr:     ls_obj = LookupServiceClient(ls_url, retry_count=5)
    YYYY-MM-DDTHH:MM:SS.475Z Wa(03)+ host-2258494   File "/usr/lib/vmware/site-packages/cis/cisreglib.py", line 314, in __init__
    YYYY-MM-DDTHH:MM:SS.475Z Wa(03)+ host-2258494

    YYYY-MM-DDTHH:MM:SS.477Z Wa(03) host-2258494 <vpxd-svcs> Service pre-start command's stderr:     return self.sslsocket_class._create(
    YYYY-MM-DDTHH:MM:SS.477Z Wa(03)+ host-2258494   File "/usr/lib/python3.10/ssl.py", line 1070, in _create
    YYYY-MM-DDTHH:MM:SS.477Z Wa(03)+ host-2258494
    YYYY-MM-DDTHH:MM:SS.477Z Wa(03) host-2258494 <vpxd-svcs> Service pre-start command's stderr:     self.do_handshake()
    YYYY-MM-DDTHH:MM:SS.477Z Wa(03)+ host-2258494   File "/usr/lib/python3.10/ssl.py", line 1341, in do_handshake
    YYYY-MM-DDTHH:MM:SS.477Z Wa(03)+ host-2258494
    YYYY-MM-DDTHH:MM:SS.477Z Wa(03) host-2258494 <vpxd-svcs> Service pre-start command's stderr:     self._sslobj.do_handshake()
    YYYY-MM-DDTHH:MM:SS.477Z Wa(03)+ host-2258494 ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: path length constraint exceeded (_ssl.c:997)

 

Cause

The problem arises because of the basicConstraint includes path Length Constraint of 0 on the Root. A pathLenConstraint of zero indicates that no intermediate CA certificates may follow in a valid certification path.

The value for pathLen in root or chain certificate can be validated as below

• • Using Openssl CLI:
    • openssl x509 -in root.cer -noout -text | grep pathlen
  • Windows:
    • Save the certificate on local machine and open using Crypto Shell Extensions

Resolution

To resolve this issue:

• Regenerate the Custom CA Root certificate without the Path Length Constraint parameter.
• (Optional) In case of other Intermediate Certificates in the chain, ensure the same parameter is not present.