Update of PNID via Appliance Management Interface (VAMI) returns Error: Failed to to start STS service
Article ID: 318470
Updated On:
Products
VMware vCenter Server
Issue/Introduction
Symptoms:
- PNID Update via Appliance Management Interface (VAMI) fails with Error: Failed to to start STS service
- /var/log/vmware/applmgmt/pnid_change.log:
2020-05-07 13:35:59,513 - pnid_utils-run_command():71 - INFO - Running command: /usr/bin/service-control --start vmware-stsd vmware-sts-idmd
2020-05-07 13:38:03,525 - pnid_utils-run_command():89 - INFO - Command exited with exitcode : 1
2020-05-07 13:38:03,525 - pnid_utils-run_command():91 - WARNING - Command failed with following error:
2020-05-07 13:38:03,525 - pnid_utils-run_command():92 - WARNING - b'2020-05-07T13:38:03.503Z RC = 1\nStdout = \nStderr = Job for vmware-stsd.service failed because the control process exited with error code. See "systemctl status vmware-stsd.service" and "journalctl -xe" for details.\n\n2020-05-07T13:38:03.504Z {\n "detail": [\n {\n "id": "install.ciscommon.command.errinvoke",\n "args": [\n "Stderr: Job for vmware-stsd.service failed because the control process exited with error code. See \\"systemctl status vmware-stsd.service\\" and \\"journalctl -xe\\" for details.\\n"\n ],\n "localized": "An error occurred while invoking external command : \'Stderr: Job for vmware-stsd.service failed because the control process exited with error code. See \\"systemctl status vmware-stsd.service\\" and \\"journalctl -xe\\" for details.\\n\'",\n "translatable": "An error occurred while invoking external command : \'%(0)s\'"\n }\n ],\n "resolution": null,\n "problemId": null,\n "componentKey": null\n}\nError executing start on service vmware-stsd. Details {\n "detail": [\n {\n "id": "install.ciscommon.service.failstart",\n "args": [\n "vmware-stsd"\n ],\n "localized": "An error occurred while starting service \'vmware-stsd\'",\n "translatable": "An error occurred while starting service \'%(0)s\'"\n }\n ],\n "resolution": null,\n "problemId": null,\n "componentKey": null\n}\nService-control failed. Error: {\n "detail": [\n {\n "id": "install.ciscommon.service.failstart",\n "args": [\n "vmware-stsd"\n ],\n "localized": "An error occurred while starting service \'vmware-stsd\'",\n "translatable": "An error occurred while starting service \'%(0)s\'"\n }\n ],\n "resolution": null,\n "problemId": null,\n "componentKey": null\n}\n'
2020-05-07 13:38:03,525 - pnid_utils-run_command():97 - INFO - STDOUT : Operation not cancellable. Please wait for it to finish...
Performing start operation on service vmware-sts-idmd...
Successfully started service vmware-sts-idmd
Performing start operation on service vmware-stsd...
2020-05-07 13:38:03,525 - pnid_utils-run_command():98 - INFO - Done running command
2020-05-07 13:38:03,526 - pnid_utils-restart_services():1443 - ERROR - 2020-05-07T13:38:03.503Z RC = 1
Stdout =
Stderr = Job for vmware-stsd.service failed because the control process exited with error code. See "systemctl status vmware-stsd.service" and "journalctl -xe" for details.
2020-05-07 13:38:03,526 - pnid_utils-update_task_status():1293 - INFO - Task : Failed to start STS Services.
2020-05-07 13:38:03,525 - pnid_utils-run_command():89 - INFO - Command exited with exitcode : 1
2020-05-07 13:38:03,525 - pnid_utils-run_command():91 - WARNING - Command failed with following error:
2020-05-07 13:38:03,525 - pnid_utils-run_command():92 - WARNING - b'2020-05-07T13:38:03.503Z RC = 1\nStdout = \nStderr = Job for vmware-stsd.service failed because the control process exited with error code. See "systemctl status vmware-stsd.service" and "journalctl -xe" for details.\n\n2020-05-07T13:38:03.504Z {\n "detail": [\n {\n "id": "install.ciscommon.command.errinvoke",\n "args": [\n "Stderr: Job for vmware-stsd.service failed because the control process exited with error code. See \\"systemctl status vmware-stsd.service\\" and \\"journalctl -xe\\" for details.\\n"\n ],\n "localized": "An error occurred while invoking external command : \'Stderr: Job for vmware-stsd.service failed because the control process exited with error code. See \\"systemctl status vmware-stsd.service\\" and \\"journalctl -xe\\" for details.\\n\'",\n "translatable": "An error occurred while invoking external command : \'%(0)s\'"\n }\n ],\n "resolution": null,\n "problemId": null,\n "componentKey": null\n}\nError executing start on service vmware-stsd. Details {\n "detail": [\n {\n "id": "install.ciscommon.service.failstart",\n "args": [\n "vmware-stsd"\n ],\n "localized": "An error occurred while starting service \'vmware-stsd\'",\n "translatable": "An error occurred while starting service \'%(0)s\'"\n }\n ],\n "resolution": null,\n "problemId": null,\n "componentKey": null\n}\nService-control failed. Error: {\n "detail": [\n {\n "id": "install.ciscommon.service.failstart",\n "args": [\n "vmware-stsd"\n ],\n "localized": "An error occurred while starting service \'vmware-stsd\'",\n "translatable": "An error occurred while starting service \'%(0)s\'"\n }\n ],\n "resolution": null,\n "problemId": null,\n "componentKey": null\n}\n'
2020-05-07 13:38:03,525 - pnid_utils-run_command():97 - INFO - STDOUT : Operation not cancellable. Please wait for it to finish...
Performing start operation on service vmware-sts-idmd...
Successfully started service vmware-sts-idmd
Performing start operation on service vmware-stsd...
2020-05-07 13:38:03,525 - pnid_utils-run_command():98 - INFO - Done running command
2020-05-07 13:38:03,526 - pnid_utils-restart_services():1443 - ERROR - 2020-05-07T13:38:03.503Z RC = 1
Stdout =
Stderr = Job for vmware-stsd.service failed because the control process exited with error code. See "systemctl status vmware-stsd.service" and "journalctl -xe" for details.
2020-05-07 13:38:03,526 - pnid_utils-update_task_status():1293 - INFO - Task : Failed to start STS Services.
- /var/log/vmware/sso/catalina.yyyy-mm-dd.log:
07-May-2020 13:36:08.245 SEVERE [localhost-startStop-1] org.apache.catalina.core.StandardContext.startInternal Error during ServletContainerInitializer processing
javax.servlet.ServletException: com.sun.xml.ws.transport.http.servlet.WSServletException: WSSERVLET11: failed to parse runtime descriptor: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'tenantServices' defined in class path resource [stsApplicationContext.xml]: Bean instantiation via constructor failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [com.vmware.identity.sts.idm.impl.IdmTenantServicesImpl]: Constructor threw exception; nested exception is java.lang.IllegalStateException: Failed to initialize CasIdmClient
at com.sun.xml.ws.transport.http.servlet.WSServletContainerInitializer.onStartup(WSServletContainerInitializer.java:70)
at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5154)
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:183)
at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:743)
at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:719)
at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:705)
at org.apache.catalina.startup.HostConfig.deployWAR(HostConfig.java:970)
at org.apache.catalina.startup.HostConfig$DeployWar.run(HostConfig.java:1841)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.sun.xml.ws.transport.http.servlet.WSServletException: WSSERVLET11: failed to parse runtime descriptor: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'tenantServices' defined in class path resource [stsApplicationContext.xml]: Bean instantiation via constructor failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [com.vmware.identity.sts.idm.impl.IdmTenantServicesImpl]: Constructor threw exception; nested exception is java.lang.IllegalStateException: Failed to initialize CasIdmClient
at com.sun.xml.ws.transport.http.servlet.WSServletContextListener.parseAdaptersAndCreateDelegate(WSServletContextListener.java:141)
at com.sun.xml.ws.transport.http.servlet.WSServletContainerInitializer.onStartup(WSServletContainerInitializer.java:65)
... 12 more
Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'tenantServices' defined in class path resource [stsApplicationContext.xml]: Bean instantiation via constructor failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [com.vmware.identity.sts.idm.impl.IdmTenantServicesImpl]: Constructor threw exception; nested exception is java.lang.IllegalStateException:Failed to initialize CasIdmClient
Caused by: org.springframework.beans.BeanInstantiationException: Failed to instantiate [com.vmware.identity.sts.idm.impl.IdmTenantServicesImpl]: Constructor threw exception; nested exception is java.lang.IllegalStateException: Failed to initialize CasIdmClient
Caused by: com.vmware.identity.idm.InvalidArgumentException: store.getExtendedIdentityStoreData().getAttributeMap() Must not be null.
javax.servlet.ServletException: com.sun.xml.ws.transport.http.servlet.WSServletException: WSSERVLET11: failed to parse runtime descriptor: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'tenantServices' defined in class path resource [stsApplicationContext.xml]: Bean instantiation via constructor failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [com.vmware.identity.sts.idm.impl.IdmTenantServicesImpl]: Constructor threw exception; nested exception is java.lang.IllegalStateException: Failed to initialize CasIdmClient
at com.sun.xml.ws.transport.http.servlet.WSServletContainerInitializer.onStartup(WSServletContainerInitializer.java:70)
at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5154)
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:183)
at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:743)
at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:719)
at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:705)
at org.apache.catalina.startup.HostConfig.deployWAR(HostConfig.java:970)
at org.apache.catalina.startup.HostConfig$DeployWar.run(HostConfig.java:1841)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.sun.xml.ws.transport.http.servlet.WSServletException: WSSERVLET11: failed to parse runtime descriptor: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'tenantServices' defined in class path resource [stsApplicationContext.xml]: Bean instantiation via constructor failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [com.vmware.identity.sts.idm.impl.IdmTenantServicesImpl]: Constructor threw exception; nested exception is java.lang.IllegalStateException: Failed to initialize CasIdmClient
at com.sun.xml.ws.transport.http.servlet.WSServletContextListener.parseAdaptersAndCreateDelegate(WSServletContextListener.java:141)
at com.sun.xml.ws.transport.http.servlet.WSServletContainerInitializer.onStartup(WSServletContainerInitializer.java:65)
... 12 more
Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'tenantServices' defined in class path resource [stsApplicationContext.xml]: Bean instantiation via constructor failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [com.vmware.identity.sts.idm.impl.IdmTenantServicesImpl]: Constructor threw exception; nested exception is java.lang.IllegalStateException:Failed to initialize CasIdmClient
Caused by: org.springframework.beans.BeanInstantiationException: Failed to instantiate [com.vmware.identity.sts.idm.impl.IdmTenantServicesImpl]: Constructor threw exception; nested exception is java.lang.IllegalStateException: Failed to initialize CasIdmClient
Caused by: com.vmware.identity.idm.InvalidArgumentException: store.getExtendedIdentityStoreData().getAttributeMap() Must not be null.
Note: The preceding log excerpts are only examples. Date, time, and environmental variables may vary depending on your environment.
Environment
VMware vCenter Server Appliance 6.7.x
Cause
The issue is caused due the following conditions:
- vCenter Server has the migration path as 5.x -> 6.0/6.5 -> 6.7
- Source vCenter Server (Windows based) was installed with shortname instead of Fully Qualified Domain Name
- LocalOS Identity source in vCenter Server Appliance (VCSA) is set to short name instead of localos
Resolution
This issue is resolved with VMware vCenter Server 6.7 Update 3j.
Workaround:
- Revert the node to a working state post PNID update failure
- Login to the VCSA via vsphere client with Single Sign-on administrator credentials
- Navigate to Administration -> Configuration
- Delete the LocalOS Identity source with vCenter machine short name
- Click on Identity Sources on the right pane
- Select the Identity Source with type value as short name of VCSA
- Click remove to clear the entry
- Create an Identity Source with Domain as localos
- Click on Add Identity Source
- Select the Identity Source Type as Local operating system of SSO server
- Enter the value under Name as localos and click Add
Feedback
Yes
No
Powered by
