VM operations hang due to expired VSM service account password
search cancel

VM operations hang due to expired VSM service account password


Article ID: 316456


Updated On:


VMware vCenter Server


On vCenter Server 8.0U2, when attempting to perform VM operations, the operation hangs and completes after an unknown amount of time or fails to complete.
  • Examples of affected VM operations:
    • Powering on
    • Rebooting
    • Cloning
    • Deployment of VMs via OVA/OVF
  • In some cases an error message is shown saying 
    Failed to clone state for the entity '<VM_Template_Name>' on extension vService Manager

    for example:
  • In /var/log/vmware/vsm/vsm.log, you may see:
    2023-11-02T23:11:57.450-04:00 INFO [Thread-4] ServiceUtil.java 137 - Acquiring SAML token for user [email protected]
    2023-11-02T23:11:57.787-04:00 ERROR [Thread-4] SoapBindingImpl.java 185 - SOAP fault
    com.sun.xml.internal.ws.fault.ServerSOAPFaultException: Client received SOAP Fault from server: Password of the user logging on is expired. :: Password of the user logging on is expired. :: User account expired: {Name: vmware-vsm-758251d3-01bc-41a9-9269-c1b8287facba, Domain: vsphere.local} Please see the server log to find more detail regarding exact cause of the failure.
    com.vmware.vim.sso.client.impl.SecurityTokenServiceImpl$RequestResponseProcessor.sendRequest(SecurityTokenServiceImpl.java:983) [libwstclient.jar:?]
          at com.vmware.vim.sso.client.impl.SecurityTokenServiceImpl$RequestResponseProcessor.executeRoundtrip(SecurityTokenServiceImpl.java:902) [libwstclient.jar:?]
          at com.vmware.vim.sso.client.impl.SecurityTokenServiceImpl.acquireToken(SecurityTokenServiceImpl.java:155) [libwstclient.jar:?]
          at com.vmware.vsm.utils.ServiceUtil.getSamlTokenForSvcAccount(ServiceUtil.java:138) [vsm.jar:?]
          at com.vmware.vsm.vc.VCenterListener.loadSamlTokenAndPrivateKey(VCenterListener.java:473) [vsm.jar:?]
          at com.vmware.vsm.vc.VCenterListener.initializeConnection(VCenterListener.java:268) [vsm.jar:?]
          at com.vmware.vsm.vc.VCenterListener.run(VCenterListener.java:301) [vsm.jar:?]
    2023-11-02T23:11:57.843-04:00 INFO [Thread-4] ServiceUtil.java 142 - Password expired. Resetting service account password
  • OVF deployment may hang at power on 
In /var/log/vmware/vpxd/vpxd.log you will see events similar to below 

2024-02-21T18:34:51.721-06:00 info vpxd[27763] [Originator@6876 sub=VmProv opID=TxId: 0625c4dc-8450-4f08-aee2-12d5add051b0-47-01] P
owering on VM '[vsanDatastore] 6b96d665-ca86-9992-f3f8-043201173960/Interconnect-C3-IX-R1.vmx' on host <host fqdn/ip>
2024-02-21T18:34:51.725-06:00 warning vpxd[27763] [Originator@6876 sub=vmomi.soapStub[258] opID=TxId: 0625c4dc-8450-4f08-aee2-12d5a
dd051b0-47-01] SOAP request returned HTTP failure; <<io_obj p:0x00007fdab0421b10, h:182, <TCP ' : 37532'>, <TCP '
 : 15007'>>, /vsm/ovfConsumer/>, method: notifyPowerOn; code: 500(Internal Server Error)
2024-02-21T18:34:51.725-06:00 info vpxd[27763] [Originator@6876 sub=OvfConsumers opID=TxId: 0625c4dc-8450-4f08-aee2-12d5add051b0-47
-01] Failed to invoke OVF stub adapter, will re-try after login; N5Vmomi5Fault13SecurityError9ExceptionE(Fault cause: vmodl.fault.S
--> )
--> [context]zKq7AVECAQAAAIUcWQEsdnB4ZAAAxbVTbGlidm1hY29yZS5zbwAAUglDAIwxRACaSEsBoy8XbGlidm1vbWkuc28AAX6eJQFZICABOk4gAZ3HH4JJ8TUBdn
  • At the same time in /var/log/vmware/vsm/vsm.log
2024-02-21T18:34:51.724-06:00 ERROR [pool-5-thread-1] VsmActivationValidator.java 267 - Failed to validate user: only vpxd-svc-acct requests allowed and not


VMware vCenter Server 8.0.2


The root cause of the problem is missing jar files from the classpath of the VSM service.


This has been fixed in vCenter 8.0 U2b. 

VMware vCenter Server 8.0 Update 2b Release Notes

As a workaround, restart the VSM service to recreate the service account. 
  1. SSH to vCenter via root
  2. Restart the VSM service:
    # service-control --restart vsm
  • Login to the VAMI of vCenter and restart the VMware Service Manager 

Note: Every time the VSM service is restarted, a new service account and password is created.