Replacing the vCenter Server's VMCA certificate with a Subordinate Certificate Authority certificate fails with the error: Error Message : Not a CA Cert

Products

VMware vCenter Server

Issue/Introduction

Symptoms:

Replacing the VMCA certificate on vCenter Server 6.0 Update 2 with a subordinate CA certificate fails.
In the Certificate Manager Utility, you see the error:

Error Message : Not a CA Cert
In the certificate-manager.log file, you see entries similar to:

YYYY-MM-DDT<time> INFO certificate-manager Replacing Root Cert using Custom CA...
</time>YYYY-MM-DDT<time> INFO certificate-manager Running command :- ['/usr/lib/vmware-vmca/bin/certool', '--rootca', '--cert', '/tmp/root_signing_chain.cer', '--privkey', '/tmp/vmca_issued_key.key', '--server', 'localhost']
</time>YYYY-MM-DDT<time> INFO certificate-manager Command output :-
</time>Error: 70011, VMCAAddRootCertificatePrivate() failedStatus : Failed
Error Code : 70011
Error Message : Not a CA Cert
YYYY-MM-DDT<time> ERROR certificate-manager Error: 70011, VMCAAddRootCertificatePrivate() failedStatus : Failed
</time>Error Code : 70011
Error Message : Not a CA Cert
YYYY-MM-DDT<time> ERROR certificate-manager Error while performing Cert Replacement operation, please see /var/log/vmware/vmcad/certificate-manager.log for more information.
</time>YYYY-MM-DDT<time> ERROR certificate-manager {
</time>"resolution": null,
"detail": [
{
"args": [
"Error: 70011, VMCAAddRootCertificatePrivate() failedStatus : Failed\nError Code : 70011\nError Message : Not a CA Cert\n"
],
"id": "install.ciscommon.command.errinvoke",
"localized": "An error occurred while invoking external command : 'Error: 70011, VMCAAddRootCertificatePrivate() failedStatus : Failed\nError Code : 70011\nError Message : Not a CA Cert\n'",
"translatable": "An error occurred while invoking external command : '%(0)s'"
},
"Error while performing certool rootca command"
],
"componentKey": null,
"problemId": null
}
YYYY-MM-DDT<time> INFO certificate-manager Performing rollback of Root Cert...</time>

The certificate-manager.log file is located at:
- Windows vCenter Server: C:\ProgramData\VMware\vCenterServer\logs\vmca\certificate-manager.log
- vCenter Server Appliance: /var/log/vmware/vmcad/certificate-manager.log
Note: The preceding log excerpts are only examples. Date, time, and environmental variables may vary depending on your environment.
Running this command returns CA:FALSE:

vCenter Server Appliance:

openssl x509 -in root_signing_cert.cer -text -noout | grep CA\:

Windows vCenter Server:

"C:\Program Files\VMware\vCenter Server\openSSL\openssl" x509 -in C:\Certs\root_signing_cert.cer -text -noout | findstr CA:

Running the openssl req -in /tmp/vmca_issued_csr.csr -noout -text | grep -A4 "Requested Extensions" command on the vCenter Server Applaince displays:

Requested Extensions:

X509v3 Subject Alternative Name:
email:[email protected], DNS:vcsa.example.com
X509v3 Subject Key Identifier:
##:##:##:##:##:##:##:##:##:##:##:##:##:##:##:##:##:##:##:##

Cause

This issue occurs when an incorrect signing request (CSR) is generated by the Certificate Manager Utility for the root signing certificate in vCenter Server 6.0 Update 2.

Notes:

In 6.0 Update 1b and earlier, the CSR is generated by copying the attributes from the default existing root cert of the VMCA which already has the required extensions on the certificate.
In 6.0 Update 2, the CSR is generated using a *.cfg file to specify company data as part of the Subject of the VMCA Root.

This change has not considered the required fields of VMCA Signing Certificate, which results in the CSR not including the Key Usage and CA:True values.

Note: For more information on key usage go to the additional information section below.

Resolution

This is a known issue affecting vCenter Server 6.0 Update 2.

To workaround this issue, manually generate the CSR using OpenSSL:

Windows vCenter Server:

Connect to vCenter Server.
Navigate to C:\Certs\ and create blank text file named vmca_root.cfg.
Add this template to the file:

[ req ]
default_bits = 2048
default_keyfile = rui.key
distinguished_name = req_distinguished_name
encrypt_key = no
prompt = no
string_mask = nombstr
req_extensions = v3_req

[ v3_req ]
basicConstraints = CA:TRUE
keyUsage = keyCertSign, cRLSign
subjectAltName = @alt_names

[ req_distinguished_name ]
countryName = US
stateOrProvinceName = NY
localityName = New York
0.organizationName = VMWare
organizationalUnitName = VMCA
commonName = vcsa.example.com
emailAddress = [email protected]

[alt_names]
DNS.1 = vcsa.example.com
IP.1 = IP_Address

Note: The subjectAltName, alt_names, DNS and IP fields are optional.
Create the CSR using this command:

openssl req -days 365 -new -newkey rsa:2048 -keyout C:\Certs\vmca.key -out C:\Certs\vmca.csr -config C:\Certs\vmca_root.cfg

Note: Change the number of days if a different expiration date is required.
Provide the resulting vmca.csr to the CA for certificate creation.

vCenter Server Appliance:

Connect to the vCenter Server Appliance.
Navigate to /tmp/.
Create the vmca_root.cfg file with this command:

touch vmca_root.cfg
Open the vmca_root.cfg using a text editor:

vi vmca_root.cfg
Add this template to the file:

[ req ]
default_bits = 2048
default_keyfile = rui.key
distinguished_name = req_distinguished_name
encrypt_key = no
prompt = no
string_mask = nombstr
req_extensions = v3_req

[ v3_req ]
basicConstraints = CA:TRUE
keyUsage = keyCertSign, cRLSign
subjectAltName = @alt_names

[ req_distinguished_name ]
countryName = US
stateOrProvinceName = NY
localityName = New York
0.organizationName = VMWare
organizationalUnitName = VMCA
commonName = vcsa.example.com
emailAddress = [email protected]
[alt_names]
DNS.1 = vcsa.example.com
IP.1 = 192.168.1.2

Note: The subjectAltName, alt_names, DNS and IP fields are optional.
Create the CSR using this command:

openssl req -days 365 -new -newkey rsa:2048 -keyout /tmp/vmca.key -out /tmp/vmca.csr -config /tmp/vmca_root.cfg

Note: Change the number of days if a different expiration date is required.
Provide the resulting vmca.csr to the CA for certificate creation.

Additional Information

The certificate signed by the CA's Key Usage is missing the required Certificate Sign and CRL Sign.

Run this command to find the Certificate Sign and CRL Sign:

openssl x509 -in rui.crt -noout -text | grep 'X509v3 Key Usage' -A1