Unable to login to vCenter Web Client as vapi-endpoint fails to start due to solution user certificate

Products

• VMware vCenter Server

Issue/Introduction

• Users will be unable to log in to the vCenter Web Client.
• The vapi-endpoint service may fail to start even though machine SSL and STS certificates appear valid.
• A banner stating "An error occurred while fetching identity providers" appears during login.
• The browser may also show a "[500] An error occurred while processing the authentication response" or "Empty SSO response string" error.
  
  
• Example output when checking the certificate expiry status:
  
  

  root@vCenter [ ~ ]# for i in $(/usr/lib/vmware-vmafd/bin/vecs-cli store list); do echo STORE $i; sudo /usr/lib/vmware-vmafd/bin/vecs-cli entry list --store $i --text | egrep "Alias|Not After"; done
  
  STORE MACHINE_SSL_CERT
  Alias : __MACHINE_CERT
              Not After : Expired date
  STORE TRUSTED_ROOTS
  Alias : 97cadaa2a7f732e1c93a9d53fbd325b24a105f53
              Not After : Expired date
  STORE TRUSTED_ROOT_CRLS
  Alias : efe52331ae2313fb1df4a53a2ed63a73a5c46f79
  STORE machine
  Alias : machine
              Not After : Expired date
  STORE vsphere-webclient
  Alias : vsphere-webclient
              Not After : Expired date
  STORE vpxd
  Alias : vpxd
              Not After : Expired date
  STORE vpxd-extension
  Alias : vpxd-extension
              Not After : Expired date
  STORE hvc
  Alias : hvc
              Not After : Expired date
  STORE data-encipherment
  Alias : data-encipherment
              Not After : Expired date
  STORE APPLMGMT_PASSWORD
  STORE SMS
  Alias : sms_self_signed
              Not After : Expired date
  STORE wcp
  Alias : wcp
              Not After : Expired date

   

• In vCenter - /var/log/vmware/sso/vmware-identity-sts.log, the following error "Solution user cert is not valid" appears.
  
  xxxx-xx-xxTxx:xx:xx.694Z INFO sts[79:tomcat-http--41] [CorId=xxxxxxxxxxxxxx-xxxx-xxxx-xxxxxxxxxxxxx] [com.vmware.identity.sts.InvalidCredentialsException] Censored exception
com.vmware.identity.sts.InvalidCredentialsException: Solution user cert is not valid.
        at com.vmware.identity.sts.auth.impl.BSTAuthenticator.checkValidCertificate(BSTAuthenticator.java:220) ~[libsts.jar:?]

Environment

vCenter 7.x

vCenter 8.x

Cause

One or more Solution User certificates have expired or are inconsistent with the SSO service.

Resolution

Replace the Solution User certificates using the vSphere Certificate Manager or the vCert utility.

• Option 1: vSphere Certificate Manager

1. 1. SSH to the vCenter Server as root.
   2. Access the bash shell: shell
   3. Launch the Certificate Manager: /usr/lib/vmware-vmca/bin/certificate-manager
   4. Select Option 6 (Replace Solution user certificates with VMCA certificates).
   5. Provide the required values for the certificate template and confirm the replacement.
      
      

• Option 2: vCert Tool (Scripted)
  
  • Use the vCert tool for an automated replacement process. Refer to KB 385107