Machine SSL certificate replacement and auto-renewal in vCenter Server 8.x

Products

• VMware vCenter Server

Issue/Introduction

Instructions for replacing expiring or expired Machine SSL certificates in vCenter Server 8.x. This document covers the manual replacement process using the vCert utility and details the automatic renewal requirements introduced in vCenter 8.0 Update 3h.

• vCenter UI displays: "Error occurred while fetching machine certificates".
  
  
• Services sps (Profile-Driven Storage) and vpxd-svcs fail to start.
  
  
• Log /var/log/vmware/vmon/vmon.log contains:
  ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: certificate has expired.

Environment

• VMware vCenter Server 8.x
• VMware Cloud Foundation 5.x / 9.x

Cause

The internal vCenter Server self-signed Machine SSL certificate (__MACHINE_CERT) has reached its expiration date.

Resolution

Replace the Machine SSL certificate using one of the following methods:

Option 1: Manual Replacement (Required for vCenter versions prior to 8.0 Update 3h)
  1.  Take a powered-off snapshot of the vCenter Server Appliance.
  2.  Establish an SSH session to the vCenter Server.
  3.  Download and upload the `vCert` utility (vCert - Scripted vCenter expired certificate replacement).
  4.  Execute the utility: `./vCert.py`.
  5.  Select Option 3 (Manage certificates) then Option 1 (Machine SSL certificate).
  6.  Restart all services: service-control --stop --all && service-control --start --all
  7.  Verify the status of all certificates on the vCenter by running the following command:
         for i in $(/usr/lib/vmware-vmafd/bin/vecs-cli store list); do echo STORE $i; sudo /usr/lib/vmware-vmafd/bin/vecs-cli entry list --store $i --text | egrep "Alias|Not After"; done

Option 2:Native Auto-Renewal (Available for vCenter 8.0 Update 3h and later)
  1.  Ensure the environment is upgraded to at least vCenter 8.0 Update 3h or VCF 9.0.2.
  2.  Verify the vCenter Advanced Setting `vpxd.certmgmt.mode` is set to `vmca`.
  3.  Machine SSL certificates will automatically renew 5 days before expiration without service interruption.
  4.  To monitor status, check the "Certificate Status" alarm in the vSphere Client.
  5.  You can verify the status of all certificates on the vCenter by running the following command:
         for i in $(/usr/lib/vmware-vmafd/bin/vecs-cli store list); do echo STORE $i; sudo /usr/lib/vmware-vmafd/bin/vecs-cli entry list --store $i --text | egrep "Alias|Not After"; done

Additional Information

Contact Broadcom Support