vCenter certificate replacement fails with "INVALID

Products

VMware vCenter Server

Issue/Introduction

Replacement of vCenter Machine SSL certificate with a custom SSL certificate fails.
Certificate replacement using vSphere client fails with "Replace cert failed: Exception found(Failed to notify applmgmt on http://localhost:1080/api/appliance/certificates/notification, on all retries)
In the var/log/vmware/certificatemanagement/certificatemanagement-svcs.log, you may find entries similar to:

[tomcat-exec-1 [] ERROR com.vmware.certificatemanagement.impl.tls.TlsReplace opId=] TLS Certificate replacement failed : Failed to notify APPLMGMT on http://localhost:1080/api/appliance/certificates/notification, on all retries

In the var/log/vmware/applmgmt/applmgmt.log, you may find entries similar to:

INFO:vmware.appliance.vapi.auth:Authorization request for service_id: com.vmware.appliance.certificates.notification, operation_id: notify
ERROR:root:Unable to authorize request with authz client: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unsuitable certificate purpose (_ssl.c:1007)

Certificate replacement using Certificate manager(/usr/lib/vmware-vmca/bin/certificate-manager) utility fails with INVALID_KEY error

ERROR certificate-manager ERROR:: INVALID_KEY, the private key doesnot match the certificate. Please provide a valid certificate and Key pair.
ERROR certificate-manager Error while replacing Machine SSL Cert, please see /var/log/vmware/vmcad/certificate-manager.log for more information.

Certificate replacement using vCert utility fails with "Operation failed: Unable to create entry _MACHINE_CERT in the VECS store MACHINE_SSL_CERT"

Environment

VMware vCenter Server 7.x
VMware vCenter Server 8.x

Cause

This issue occurs due to any of the following reasons:

The private key does not match the certificate.
The certificate contains incorrect key usage extensions

Resolution

Validating Certificate and Private key:

Verify if the certificate matches the private key by comparing the sha256 hash of custom certificate and private key using below commands:

openssl x509 -noout -modulus -in custom.crt | openssl sha256

openssl rsa -noout -modulus -in private.key | openssl sha256

Note: custom.crt is the custom vCenter certificate and private.key is the private key. If the CSR is generated from the vSphere client, private key can be obtained using /usr/lib/vmware-vmafd/bin/vecs-cli entry getkey --store MACHINE_SSL_CERT --alias __MACHINE_CSR

The output of both the above commands should match

Note: If the output of above commands do not match, the custom certificate is not created from the CSR generated on the vCenter. To resolve this issue, generate a new CSR from the vCenter and obtain a certification from the CA using the generated CSR.

Validating Extended Key Usage Extensions:

X509v3 Extended Key Usage section of the certification should not contain TLS Web Server Authentication, TLS Web Client Authentication or both.
Review the certificate using the below command.

openssl x509 -in custom.crt -text -noout

Note: If the Extended Key Usage section contains the above mentioned entries, the certificate needs to be recreated by the CA.

Additional Information

If Microsoft Certificate authority is being used refer Creating a Microsoft Certificate Authority Template for SSL certificate creation in vSphere