Verify and resolve expired vCenter Server certificates using command line interface

Products

VMware vCenter Server

Issue/Introduction

This article provides steps to verify certificate expiration dates and resolve expired certificates in the vCenter Server using the command line interface.

Warnings in the vCenter interface showing certificates are expiring soon.
Errors seen:

503 service not available...endpoint

or

no healthy upstream

Environment

VMware vCenter Server 8.0
VMware vCenter Server 6.x
VMware vCenter Server 7.0.x
VMware vCenter Server Appliance 6.7.x
VMware vCenter Server Appliance 6.5.x

Cause

This issue is seen when one or more required certificates are expired or will expire soon in the vCenter Server.

Resolution

Verify certificate expiration date

Check the Single Sign-on Token Signing (STS) certificate, see Checking Expiration of STS Certificate on vCenter Server.
Run the below commands to see the status of the environments certificates:

Run this command on the vCenter Appliance:

for store in $(/usr/lib/vmware-vmafd/bin/vecs-cli store list | grep -v TRUSTED_ROOT_CRLS); do echo "[*] Store :" $store; /usr/lib/vmware-vmafd/bin/vecs-cli entry list --store $store --text | grep -ie "Alias" -ie "Not After";done;

Run this command on the Windows vCenter Server:

$VCInstallHome = [System.Environment]::ExpandEnvironmentVariables("%VMWARE_CIS_HOME%");foreach ($STORE in & "$VCInstallHome\vmafdd\vecs-cli" store list){Write-host STORE: $STORE;& "$VCInstallHome\vmafdd\vecs-cli" entry list --store $STORE --text | findstr /C:"Alias" /C:"Not After"}

You will see an output similar to:

Ensure the dates are in the future.

Resolving expired certificates

Caution:

Backup or create a virtual machine snapshot before proceeding.
It is recommended power off all linked external Platform Services Controllers/vCenter Servers with embedded PSCs at the same time and to take a snapshot of every linked node VM.

Custom certificates

If you have expired trusted root or SSL certificates it is recommended to get the system working again using the default VMware Certificate Authority certificates, then to re-apply your custom certificate, see Replacing a vSphere 6.x /7.x Machine SSL certificate with a Custom Certificate Authority Signed Certificate

STS certificate

For vCenter with embedded PSC, or external PSCs only, do the following only on one node for each system of linked nodes: replace the STS certificate per "Signing certificate is not valid" error in VCSA 6.5.x/6.7.x and vCenter Server 7.0.x

Trusted root certificate

For vCenter with embedded PSC, or external PSCs only, do the following once in a system of linked nodes: Run certificate-manager per How to use vSphere Certificate Manager to Replace SSL Certificates, and use Option 4 to generate a new root certificate and replace all certificates.
On all remaining vCenter and PSCs in the linked system, do the following:

Run certificate-manager option 3 to replace the Machine SSL certificate
Run certificate-manager option 6 to replace the solution user certificates

Machine SSL certificate

On each node (vCenter, vCenter with embedded PSC, or external PSC) found with this expired certificate, run certificate-manager option 3 to replace the SSL certificate.

Solution user certificates

If one or more of these has expired, On each node (vCenter, vCenter with embedded PSC, or external PSC) found with this expired certificate, run certificate-manager option 6 to replace the solution users certificates.

Note: If option 3 or 6 of the Certificate manager fails for the VCenter you could try using option 8 to reset all Certificates.