Expired vCenter CA certificate

Products

VMware vCenter Server

Issue/Introduction

This article provides the steps to be followed when renewing CA certificate on vCenter.

In case the custom SSL certificate is already expired ,any of the following errors/issues may be observed while accessing the vCenter UI:

Certificate Status alert alarm
HTTP Status 500 – Internal Server Error
No healthy upstream
Some services are not starting

After the certificate replacement has been attempted in a case where it has not yet expired, if the process to take the vCenter snapshot was performed while vCenter HA is still active, vCenter will appear to be in a down state after a reboot, and the IP address assigned to the eth0 interface will not respond to pings.

Environment

vCenter 7.x

vCenter 8.x

Resolution

Ensure you take a powered-off snapshot of the vCenter Server for backup:

If the vCenter is using vCenter High Availability (VCHA), you must destroy the VCHA configuration before taking the snapshot.
If vCenter is using vCenter HA (VCHA) then you must Remove VCHA before taking the snapshot.
- Remove vCenter High Availability
  - From the vSphere Client:
    - In the vSphere UI, navigate to Configure > vCenter HA.
    - Click the REMOVE VCHA button
    - In the pop-up dialog box, check the option "Delete Passive and Witness node VMs".
  - If vSphere Client is unavailable:
    - Power down the current witness and passive nodes from the respective residing ESXi Host clients and unregister/delete them
- Clean Up from the Active Node via SSH
  - SSH into the vCenter Server Appliance (Active Node).
  - Run the following command to complete the removal:(For 7.0 vCenter and above)

vcha-destroy -f

If the vCenter is part of an Enhanced Linked Mode (ELM) configuration, you must power down all linked vCenter Servers and external Platform Services Controllers (PSCs) simultaneously. Once all nodes are powered off, take a powered-off snapshot of each node. For more information see KB: VMware vCenter in Enhanced Linked Mode pre-changes snapshot (online or offline) best practice

Caution: Do not take snapshots while vCenter HA is enabled. Before Enabling vCenter HA you must first Delete All snapshots.

Renew the expired certs using vCert tool:

Download vCert script from KB vCert - Scripted vCenter Expired Certificate Replacement.
Run vCert option 1.
Take note of the expired CA certificate ID.
Select option (Manage certificates).
Select option (VMCA certificate).
Select vCert option (Replace the VMCA certificate using a self-signed certificate and regenerate all certificates).
Restart all vCenter services.

service-control --stop --all && service-control --start --all

Note: If other certificates are still expired then all services may not start yet even though you have replaced the expired CA certificate.

Additional Information

Verify an expired CA certificate is not in use

Obtain the CA certificate Subject Key Identifier:
1. Use vCert option 1:
2. Obtain with command:/usr/lib/vmware-vmafd/bin/vecs-cli entry list --store TRUSTED_ROOTS --text
Run the command for each certificate to verify if it is using the expired CA certificate. The Authority Key Identifier will match the Subject Key Identifier of the CA certificate in use.
/usr/lib/vmware-vmafd/bin/vecs-cli entry list --store MACHINE_SSL_CERT --text | more
/usr/lib/vmware-vmafd/bin/vecs-cli entry list --store machine --text | more
/usr/lib/vmware-vmafd/bin/vecs-cli entry list --store vpxd --text | more
/usr/lib/vmware-vmafd/bin/vecs-cli entry list --store vpxd-extension --text | more
/usr/lib/vmware-vmafd/bin/vecs-cli entry list --store vsphere-webclient --text | more
/usr/lib/vmware-vmafd/bin/vecs-cli entry list --store SMS --text | more
/usr/lib/vmware-vmafd/bin/vecs-cli entry list --store hvc --text | more
/usr/lib/vmware-vmafd/bin/vecs-cli entry list --store data-encipherment --text | more
/usr/lib/vmware-vmafd/bin/vecs-cli entry list --store wcp --text | more /usr/lib/vmware-vmafd/bin/vecs-cli entry list --store TRUSTED_ROOT_CRLS --text | more

Remove a vCenter HA Configuration