Verify and resolve expired vCenter Server certificates using the command line interface

Products

VMware vCenter Server

Issue/Introduction

This article provides steps on how to verify certificate expiration dates and resolve expired certificates in the vCenter Server using the command line interface.

Warnings in the vCenter interface showing certificates are expiring soon.
The following errors may be observed:

503 service not available...endpoint

or

no healthy upstream

or

[500] An error occurred while fetching identity providers

Environment

VMware vCenter Server

Cause

This issue is seen when one or more required certificates are expired or will expire soon in the vCenter Server.

Resolution

Verify certificate expiration date

Check the Single Sign-on Token Signing (STS) certificate; see Checking Expiration of STS Certificate on vCenter Server.
Run the below commands to see the status of the environmental certificates:

Run this command on the vCenter Appliance:

for store in $(/usr/lib/vmware-vmafd/bin/vecs-cli store list | grep -v TRUSTED_ROOT_CRLS); do echo "[*] Store :" $store; /usr/lib/vmware-vmafd/bin/vecs-cli entry list --store $store --text | grep -ie "Alias" -ie "Not After";done;

Run this command on the Windows vCenter Server:

$VCInstallHome = [System.Environment]::ExpandEnvironmentVariables("%VMWARE_CIS_HOME%");foreach ($STORE in & "$VCInstallHome\vmafdd\vecs-cli" store list){Write-host STORE: $STORE;& "$VCInstallHome\vmafdd\vecs-cli" entry list --store $STORE --text | findstr /C:"Alias" /C:"Not After"}

Similar output:

Ensure the dates are in the future.

Resolving expired certificates

Caution:

Take a backup or create a virtual machine snapshot before proceeding.
It is recommended to power off all linked external Platform Services Controllers/vCenter Servers with embedded PSCs at the same time and to take a snapshot of every linked node VM.

Custom certificates

If there are expired trusted root or SSL certificates, to get the system working again, it is recommended to use the default VMware Certificate Authority certificates. The VMCA signed certificate can then be replaced with the custom certificate; see Replacing a vSphere 6.x /7.x Machine SSL certificate with a Custom Certificate Authority Signed Certificate

STS certificate

For vCenter with embedded PSC, or external PSCs only, do the following only on one node for each system of linked nodes: replace the STS certificate per "Signing certificate is not valid" error in VCSA 6.5.x/6.7.x and vCenter Server 7.0.x

Trusted root certificate

For vCenter with embedded PSC, or external PSCs only, do the following once in a system of linked nodes: Run certificate-manager per How to use vSphere Certificate Manager to Replace SSL Certificates, and use Option 4 to generate a new root certificate and replace all certificates.
On all remaining vCenter and PSCs in the linked system, do the following:

Run certificate-manager option 3 to replace the Machine SSL certificate
Run certificate-manager option 6 to replace the solution user certificates

Machine SSL certificate

On each node (vCenter, vCenter with embedded PSC, or external PSC) found with this expired certificate, run certificate-manager option 3 to replace the SSL certificate.

Solution user certificates

If one or more of these has expired, on each node (vCenter, vCenter with embedded PSC, or external PSC) found with this expired certificate, run certificate-manager option 6 to replace the solution users' certificates.

Note: If option 3 or 6 of the Certificate manager fails for the vCenter, try using option 8 to reset all Certificates.