vCenter Server certificate store guide

Products

• VMware vCenter Server 7.0
• VMware vCenter Server 8.0

Issue/Introduction

In vCenter Server, there are indeed various types of certificates, generally divided into two main categories based on management:

• VMware Endpoint Certificate Store (VECS)
• Security Token Service (STS)

To better understand the topology of certificate, refer the below picture:

This article explains the certificate stores in VECS. The VECS serves as a local repository for certificates, private keys, and other certificate information that can be stored in a keystore, which runs as part of the VMware Authentication Framework Daemon (VMAFD). 

To view all the store lists, run the command in vCenter Server with SSH access:

# /usr/lib/vmware-vmafd/bin/vecs-cli store list 
--------Default Stores-------
MACHINE_SSL_CERT
TRUSTED_ROOTS
TRUSTED_ROOT_CRLS
machine
vsphere-webclient
vpxd
vpxd-extension
hvc
data-encipherment
APPLMGMT_PASSWORD
SMS
wcp
------Non Default Stores------
BACKUP_STORE
STS_INTERNAL_SSL_CERT
KMS_ENCRYPTION

NOTE:

• STS certificate is not stored in VECS. Check and renew STS certificate with Fixcerts or Fixsts.sh
• ESXi host certificate signed by VMCA of vCenter Server is stored locally on each host and not in VECS.
  Renew ESXi host certificate, check Renew or Refresh ESXi Certificates
• To check all certificates validation refer Verify and resolve expired vCenter Server certificates using command line interface

Environment

VMware vCenter Server 7.0.x
VMware vCenter Server 8.0.x

Resolution

Delineation of each store certificate, usage and renew options:

Store	Used for	Certificate renew options
MACHINE_SSL_CERT	Store the certificate used by the reverse proxy service by exposing port 443. SSL connections to individual vCenter services always go to the reverse proxy.  Used by the VMware Directory Service (VMDIR).	vSphere UI: Renew Certificates Using the vSphere Client Fixcerts script: fixcerts Certificate Manager utility: certificate-manager
TRUSTED_ROOTS	Contains all trusted root certificates. Sync with VMDIR every 5 minutes.  Sync with all nodes in Linked Mode. To remove of a root certificate completely, it has to be unpublished from VMDIR.	Fixcerts script: fixcerts Certificate Manager utility: certificate-manager
TRUSTED_ROOT_CRLS	Store VMCA published Trusted Root certificate revocation list. The number of certificates in this store always equal to the number of certificates in TRUSTED_ROOTS.	NOTE: renewed automatically, no manual interaction.
machine	Store the solution user machine-<machine-id> certificate for authentication with vCenter Single Sign-On (SSO). Used by component manager, license server, and the logging service etc.	Fixcerts script: fixcerts Certificate Manager utility: certificate-manager
vsphere-webclient	Store the solution user vsphere-webclient-<machine-id> certificate for authentication with SSO. Used by vapi-endpoint, vsphere-ui, perfcharts etc.
vpxd	Store the solution user vpxd-<machine-id> certificate for authentication with SSO. Used by services of vpxd, api-proxy etc.
vpxd-extension	Store the solution user vpxd-extension-<machine-id> certificate for authentication with SSO. Used by services of sms, eam, content library etc.
hvc	Store the solution user hvc-<machine-id> certificate for authentication with SSO. Used when configured in Hybrid vCenter with vCenter Cloud Gateway, such as VMware Cloud on AWS.
wcp	Store the solution user wcp-<machine-id> certificate for authentication with SSO. Used by VMware vSphere with VMware Tanzu, and used for vSphere Cluster Services.
data-encipherment	Store certificate for guest customization specification password encryption.	Fixcerts script: fixcerts Fix_encipherment_cert script: fix_encipherment_cert.sh
APPLMGMT_PASSWORD	Store secret key for password encryption when scheduled backup taken in VAMI with Encrypt database password specified.	NOTE: not need renew
SMS	Store certificate for Storage Management Service (SMS). SMS is used to manages VASA (VMware API's for Storage Awareness) providers delivered by storage vendors. Storage Profile Based Management Service (SPBM) relies on SMS to communicate with VASA providers.	Fixcerts script: fixcerts
BACKUP_STORE	This store created only when used certificate manager util to replace certificate to support revert. Only the most recent state is stored as a backup. Stores the backup of the Machine_SSL certificate and the Solution User certificates.	NOTE: not need renew
STS_INTERNAL_SSL_CERT	This store created only when the vCenter Server was migrated from 5.5/6.0 The Lookupservice uses this certificate.	Fixcerts script: fixcerts
KMS_ENCRYPTION	This store created only when configure Key Management for virtual machine encryption. Store machine certificate of the KMS Server.	VECS utility: vecs-cli

 

Common Symptoms when certificates expired:

• "Signing certificate is not valid" error in VCSA 6.5.x/6.7.x and vCenter Server 7.0.x
• Alarm: STS Signing Certificates are about to expire
• Alarm: Certificate Status
• Unable to login vCenter UI with error message "Invalid Credentials"