Upgrade to vCenter 7.0 fails with "CollectRequirements for com.vmware.sso times out"
Article ID: 326251
Updated On:
Products
VMware vCenter Server
Issue/Introduction
Symptoms:
vCenter Upgrade to 7.0 fails with an error:
Upgrade phase timed out. The time planned for the upgrade phase was 15 minutes. The upgrade phase has already been running for 15 minutes.
CollectRequirements for com.vmware.sso times out
In the Tomcat webapp log [catalina.xxxx-xx-xx.log] you see the following error:
YYYY-MM-DDT16:01:10.077Z SEVE org.apache.catalina.core.StandardService Failed to initialize connector [Connector[com.vmware.identity.tomcat.VECSAwareHttp11NioProtocol-7444]]
org.apache.catalina.LifecycleException: Protocol handler initialization failed
at org.apache.catalina.connector.Connector.initInternal(Connector.java:1078)
at org.apache.catalina.util.LifecycleBase.init(LifecycleBase.java:136)
...
Caused by: java.io.IOException: Could not get key with alias __MACHINE_CERT from VECS key store
at com.vmware.identity.tomcat.VECSAwareSSLImplementation.getTransientKeyStore(VECSAwareSSLImplementation.java:162)
at com.vmware.identity.tomcat.VECSAwareSSLImplementation$1.getKeyManagers(VECSAwareSSLImplementation.java:65)
at org.apache.tomcat.util.net.SSLUtilBase.createSSLContext(SSLUtilBase.java:245)
at org.apache.tomcat.util.net.AbstractJsseEndpoint.createSSLContext(AbstractJsseEndpoint.java:98)
... 19 more
When reviewing the content in the STS_INTERNAL_SSL_CERT store in the VECS you find that it contains no entries:
# /usr/lib/vmware-vmafd/bin/vecs-cli entry list --store STS_INTERNAL_SSL_CERT
Number of entries in store : 0
Customer may experience this during vCenter upgrade version 7.0.3 to 8.0.3 fails with an error:
A problem occurred while Starting VMware Security Token Service.
Encountered an internal error. Traceback (most recent call last): File "/usr/lib/vmidentity/firstboot/vmidentity-firstboot.py", line 1170, in main vmidentityFB.boot() File "/usr/lib/vmidentity/firstboot/vmidentity-firstboot.py", line 275, in boot self.configureSTS(self.__stsRetryCount, self.__stsRetryInterval) File "/usr/lib/vmidentity/firstboot/vmidentity-firstboot.py", line 791, in configureSTS self.startSTSService() File "/usr/lib/vmidentity/firstboot/vmidentity-firstboot.py", line 751, in startSTSService returnCode = self.startService(self.__sts_service_name) File "/usr/lib/vmidentity/firstboot/vmidentity-firstboot.py", line 80, in startService update_services_runstate("start", None, False, False, svc_names=[svc_name]) File "/usr/lib/vmware/site-packages/cis/svcsController.py", line 1122, in update_services_runstate _update_services_runstate_svclist('start', svc_nodenames, File "/usr/lib/vmware/site-packages/cis/svcsController.py", line 883, in _update_services_runstate_svclist controller.start_svc(svc_id, explicit_op=explicit_op) File "/usr/lib/vmware/site-packages/cis/svcsController.py", line 516, in start_svc service_start(svc_id, quiet=_quiet, File "/usr/lib/vmware/site-packages/cis/utils.py", line 1173, in service_start raise ServiceStartException(svc_name) cis.exceptions.ServiceStartException: { "detail": [ { "id": "install.ciscommon.service.failstart", "translatable": "An error occurred while starting service 'Traceback (most recent call last): File "/usr/lib/vmidentity/firstboot/vmidentity-firstboot.py", line 1170, in main vmidentityFB.boot() File "/usr/lib/vmidentity/firstboot/vmidentity-firstboot.py", line 275, in boot self.configureSTS(self.__stsRetryCount, self.__stsRetryInterval) File "/usr/lib/vmidentity/firstboot/vmidentity-firstboot.py", line 791, in configureSTS self.startSTSService() File "/usr/lib/vmidentity/firstboot/vmidentity-firstboot.py", line 751, in startSTSService returnCode = self.startService(self.__sts_service_name) File "/usr/lib/vmidentity/firstboot/vmidentity-firstboot.py", line 80, in startService update_services_runstate("start", None, False, False, svc_names=[svc_name]) File "/usr/lib/vmware/site-packages/cis/svcsController.py", line 1122, in update_services_runstate _update_services_runstate_svclist('start', svc_nodenames, File "/usr/lib/vmware/site-packages/cis/svcsController.py", line 883, in _update_services_runstate_svclist controller.start_svc(svc_id, explicit_op=explicit_op) File "/usr/lib/vmware/site-packages/cis/svcsController.py", line 516, in start_svc service_start(svc_id, quiet=_quiet, File "/usr/lib/vmware/site-packages/cis/utils.py", line 1173, in service_start raise ServiceStartException(svc_name) cis.exceptions.ServiceStartException: { "detail": [ { "id": "install.ciscommon.service.failstart", "translatable": "An error occurred while starting service '%(0)s'", "args": [ "sts" ], "localized": "An error occurred while starting service 'sts'" } ], "componentKey": null, "problemId": null, "resolution": null } '", "args": [ "sts" ], "localized": "An error occurred while starting service 'sts'" } ], "componentKey": null, "problemId": null, "resolution": null }
Resolution
This is an unrecoverable error, please retry install. If you encounter this error again, please search for these symptoms in the VMware Knowledge Base for any known issues and possible resolutions. If none can be found, collect a support bundle and open a support request.
Environment
VMware vCenter Server 7.0.x
Cause
When vCenter Server was upgraded from an older version in the past, the installer expects to find an STS_INTERNAL_SSL_CERTIFICATE store in the VECS, containing the machine SSL certificate.
If this store does not exist, or holds no entries, the vCenter installer will fail to verify the current STS certificate during the pre-check phase and will get stuck.
As a result, when its default timeout is reached, the upgrade fails with the error shown above.
Resolution
To solve this, publish the current machine SSL certificate to the STS_INTERNAL_SSL_CERT store using the following steps:
- ensure that you have a current backup of the vCenter
- open an SSH connection to the old vCenter Server Appliance
- create an export folder:
# mkdir /certs - export the current machine SSL certificate and public key from the MACHINE_SSL_CERT store:
# /usr/lib/vmware-vmafd/bin/vecs-cli entry getcert --alias __MACHINE_CERT --store MACHINE_SSL_CERT >/certs/__MACHINE_CERT.crt # /usr/lib/vmware-vmafd/bin/vecs-cli entry getkey --alias __MACHINE_CERT --store MACHINE_SSL_CERT >/certs/__MACHINE_CERT.key - publish the certificate as a new entry called __MACHINE_CERT in the STS_INTERNAL_CERT store:
# /usr/lib/vmware-vmafd/bin/vecs-cli entry create --store STS_INTERNAL_SSL_CERT --alias __MACHINE_CERT --cert /certs/__MACHINE_CERT.crt --key /certs/__MACHINE_CERT.key - Initiate the upgrade process again.
Feedback
Yes
No
Powered by
