After Updating the Machine SSL, we can no longer turn on DRS

Products

• VMware vSphere ESXi
• VMware vCenter Server

Issue/Introduction

After upgrading the Machine SSL cert, we are unable to: 

• Turn on DRS
• Create new VMs
• Create VM templates
• Migrate VMs

You may also see the following error when you try to turn on DRS: 

Failed to log into [uri=http://localhost:1080/wcp, sessionMgr=VapiSessionManagerInfo [_sessionMgrSvcId=com.vmware.cis.session, _loginOpId=create, _logoutOpId=delete], ssoDomain=vsphere.local(#####-####-####-####-############)]: com.vmware.vapi.std.errors.internal_server_error => {data=<unset>, error_type=INTERNAL_SERVER_ERROR, messages=[com.vmware.vapi.std.localizable_message => {args=[a general error occurred while exchanging the token], default_message=Internal server error occured on authorization: a general error occurred while exchanging the token, localized=<unset>, id=vapi.security.authorization.internal_server_error, params=<unset>}]}

Finally, when you go the certificate management view in vCenter you will see that there is an expired trusted root: 

Environment

• VMware ESXi 8.0.x
• VMware vCenter Server 8.0.x
• VMware ESXi 7.0.x
• VMware vCenter Server 7.0.x

Cause

This happens when there is an expired Certificate Authority. This causes vCenter to be unable to recognize the Machine SSL as valid. This prevents DRS or vCenter tasks from running. 

Resolution

Before running vCert make sure to take a powered off snapshot of vCenter. 

• • If vCenter is linked using Enhanced Linked Mode then you must power down every vCenter and external PSC that the vCenter is linked to at the same time as the vCenter.  Once all linked vCenters and PSCs are shutdown take a powered off snapshot of each node.
    • For more information see KB: VMware vCenter in Enhanced Linked Mode pre-changes snapshot (online or offline) best practice
  • If vCenter is using vCenter HA (VCHA) then you must destroy VCHA before taking the snapshot.

1. Re-issue the CA that expired
2. Download and run vCert script from KB vCert - Scripted vCenter Expired Certificate Replacement.
   • Use utility to copy over vCert script to vCenter.
   • Unzip vCert script.
     • unzip vCert-#.#.#-########.zip
   • Change directory to the vCert folder.
     • cd vCert-#.#.#-########
   • Run vCert.
     • ./vCert.py
3. Select option 3 (Manage certificates).
4. Select option 3 (CA certificates in VMware Directory).
5. Select option 1 (to publish CA certificates to VMware Directory.).
6. Exit vCert 
7. Restart all vCenter services.
   • #service-control --stop --all && service-control --start --all

Additional Information

For additional information on vCert see: https://knowledge.broadcom.com/external/article/385107