vapi-endpoint fails to start due to solution user certificate

Products

• VMware vCenter Server

Issue/Introduction

The vapi-endpoint service fails to start even though the machine_ssl, solution users, and STS certificates are not expired.

example output when checking the certificate expiry status:

 

root@vcsa01 [ ~ ]# for i in $(/usr/lib/vmware-vmafd/bin/vecs-cli store list); do echo STORE $i; sudo /usr/lib/vmware-vmafd/bin/vecs-cli entry list --store $i --text | egrep "Alias|Not After"; done
STORE MACHINE_SSL_CERT
Alias : __MACHINE_CERT
            Not After : Expired date
STORE TRUSTED_ROOTS
Alias : 97cadaa2a7f732e1c93a9d53fbd325b24a105f53
            Not After : Expired date
STORE TRUSTED_ROOT_CRLS
Alias : efe52331ae2313fb1df4a53a2ed63a73a5c46f79
STORE machine
Alias : machine
            Not After : Expired date
STORE vsphere-webclient
Alias : vsphere-webclient
            Not After : Expired date
STORE vpxd
Alias : vpxd
            Not After : Expired date
STORE vpxd-extension
Alias : vpxd-extension
            Not After : Expired date
STORE hvc
Alias : hvc
            Not After : Expired date
STORE data-encipherment
Alias : data-encipherment
            Not After : Expired date
STORE APPLMGMT_PASSWORD
STORE SMS
Alias : sms_self_signed
            Not After : Expired date
STORE wcp
Alias : wcp
            Not After : Expired date

We get the below error in  vmware-identity-sts.log

YYYY-MM-DDThh:mm:ss.735Z INFO sts[82:tomcat-http--48] [CorId=a6696c76-029d-49f4-97be-ebc596fb4be6] [com.vmware.identity.sts.InvalidCredentialsException] Censored exception
com.vmware.identity.sts.InvalidCredentialsException: Solution user cert is not valid.
        at com.vmware.identity.sts.auth.impl.BSTAuthenticator.checkValidCertificate(BSTAuthenticator.java:222) ~[sts-7.0.0.jar:?]
        at com.vmware.identity.sts.auth.impl.BSTAuthenticator.doAuthenticate(BSTAuthenticator.java:119) ~[sts-7.0.0.jar:?]
        at com.vmware.identity.sts.auth.impl.BSTAuthenticator.authenticate(BSTAuthenticator.java:86) ~[sts-7.0.0.jar:?]
        at com.vmware.identity.sts.auth.impl.CompositeAuthenticator.authenticate(CompositeAuthenticator.java:54) ~[sts-7.0.0.jar:?]
        at com.vmware.identity.sts.auth.impl.CompositeAuthenticatorPerformanceDecorator$1.call(CompositeAuthenticatorPerformanceDecorator.java:68) ~[sts-7.0.0.jar:?]
        at com.vmware.identity.sts.auth.impl.CompositeAuthenticatorPerformanceDecorator$1.call(CompositeAuthenticatorPerformanceDecorator.java:65) ~[sts-7.0.0.jar:?]
        at com.vmware.identity.performanceSupport.PerformanceDecorator.exec(PerformanceDecorator.java:54) ~[vmware-identity-idm-interface-7.0.0.jar:?]
        at com.vmware.identity.sts.auth.impl.CompositeAuthenticatorPerformanceDecorator.authenticate(CompositeAuthenticatorPerformanceDecorator.java:65) ~[sts-7.0.0.jar:?]
        at com.vmware.identity.sts.impl.STSImpl.issue(STSImpl.java:158) ~[sts-7.0.0.jar:?]
        at com.vmware.identity.sts.impl.MultiTenantSTSImpl.issue(MultiTenantSTSImpl.java:60) ~[sts-7.0.0.jar:?]

Environment

vCenter 7.0 U3

Cause

Solution user cert is not valid

Resolution

Open an SSH session to the vCenter server
Login to the shell using the below command:
shell

launch the vSphere Certificate Manager, and execute the following commands:

/usr/lib/vmware-vmca/bin/certificate-manager

Select option 6 to reset solution user certificates with VMCA certificates

Fill in the template and proceed with the process to re-generate the solution user certificate.