Certificate Status Alarm - Clearing BACKUP_STORES certificates in the VCSA

Products

• VMware vCenter Server

Issue/Introduction

• The following alarm appears on the vCenter Server object in the vSphere Client:

         

• The vCenter certificates are verified as valid or recently regenerated.
  
  
• Reviewing the certificate stores reveals expired certificates within BACKUP_STORE or BACKUP_STORE_H5C in VECS.
   
  • Run the below command to see the status of the environmental certificates:
    
    for store in $(/usr/lib/vmware-vmafd/bin/vecs-cli store list | grep -v TRUSTED_ROOT_CRLS); do echo "[*] Store :" $store; /usr/lib/vmware-vmafd/bin/vecs-cli entry list --store $store --text | grep -ie "Alias" -ie "Not After";done;

Environment

• VMware vCenter Server 6.x
• VMware vCenter Server 7.x
• VMware vCenter Server 8.x

Cause

Replacing certificates in the vCenter Server Appliance creates two backup stores: BACKUP_STORE and BACKUP_STORE_H5C. During the replacement process, vCenter saves old certificates to these stores to allow for a rollback. If any entry in these stores expires or approaches expiration, the vSphere Client displays a Certificate Status Alarm.

Resolution

The script previously attached to this KB is deprecated.

Option 1:

Use the new, improved certificate management tool, vCert - Scripted vCenter Expired Certificate Replacement, for all certificate management and replacement workflows.

1. Take a snapshot or backup of the vCenter Server. (Note: In an Enhanced Linked Mode environment, powered-off snapshots of all vCenter Servers in the SSO domain are required).
2. Download the vCert script on the vCenter Server Appliance as described in Installation Section.
3. From the Manage Certificates menu, select Option 12: Clear expired certificates in BACKUP_STORE in VECS to remove the expired certificates from the backup stores.

Note:

• Use WinSCP to upload the vCert tool to the VCSA. For troubleshooting details, see KB article Connecting to vCenter Server Virtual Appliance using WinSCP fails with the error: Received too large (1433299822 B) SFTP packet. Max supported packet size is 1024000 B.
• vCenter Server 7.0 Update 3 and 8.0 resolve this issue so that an expired BACKUP_STORE certificate no longer triggers an alarm. However, an expired certificate in BACKUP_STORE_H5C still triggers the alarm, requiring deletion to clear the alert.

If the automated process fails to clear BACKUP_STORE, perform the following manual steps:

 

Option 2:

1. Take a snapshot or backup of the vCenter Server. (Note: In an Enhanced Linked Mode environment, powered-off snapshots of all vCenter Servers in the SSO domain are required).
2. Access the vCenter Server Appliance via SSH as root.
3. Run the following command to list the store contents:
/usr/lib/vmware-vmafd/bin/vecs-cli entry list --store BACKUP_STORE
4. Note the alias of the certificate to remove from BACKUP_STORE.
5. Run the following command to remove the expired certificate:
   /usr/lib/vmware-vmafd/bin/vecs-cli entry delete --store BACKUP_STORE --alias <cert_alias>