SDDC Manager UI fails to load after reboot when vCenter root password has expired
search cancel

SDDC Manager UI fails to load after reboot when vCenter root password has expired

book

Article ID: 305970

calendar_today

Updated On: 04-09-2025

Products

VMware Cloud Foundation

Issue/Introduction

  • SDDC Manager UI fails to load after reboot when vCenter root password has expired
  • SDDC Manager UI log /var/log/vmware/vcf/sddc-manager-ui/sddcManagerserver.log will report the below errors
    Caught ssh exec error event
    WARNING: Your password has expired
  • SSH attempt to vCenter prompts to change the root password

    Sample output
    login as: root
    Pre-authentication banner message from server:
    |
    | VMware vCenter Server 8.0.2.00200
    |
    | Type: vCenter Server with an embedded Platform Services Controller
    |
    End of banner message from server
    Keyboard-interactive authentication prompts from server:
    | Password:
    End of keyboard-interactive prompts from server
    Last login: Fri Sep 27 22:53:21 2024 from x.x.x.x
    sudo: Account or password is expired, reset your password and try again
    Changing password for root.
    Current password:



Environment

VMware Cloud Foundation 4.x , 5.x

Cause

On boot, the SDDC tries to establish an SSH connection to the Management Domain vCenter. If the root password on the Management Domain vCenter has expired this connection will fail.

Even if the root password is changed on the Management Domain vCenter side, the UI will still not launch correctly as SDDC Manager is unaware of the new root password on the vCenter side and will continue to try to connect using the old password.

This in turn will lock the new, valid Management Domain vCenter root password due to an excessive number of failed logins.

Resolution

 
Change the root password on the Management domain vCenter and update the SDDC Manager with those new credentials
 
 
Steps to follow:
 
  1. SSH to vCenter and login with root. Set a new password on the Password change prompt
  2. Take snapshot of SDDC Manager VM
  3. SSH to SDDC Manager with vcf and su to root
  4. Run the below command to list the vcenter details from platform and note down the id of the Management vCenter.
    psql -h localhost -U postgres -d platform -c "select vm_hostname,id,status from vcenter"

    Sample output

       vm_hostname     |                  id                  | status
    ---------------------+--------------------------------------+--------
     vcsa01.example.com | xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxx602f | ACTIVE
  5. List the credential details for all vCenters manged by SDDC
    curl 'localhost/credentials-service/credentials?entityType=VCENTER&credentialType=SSH' | json_pp

    Note: The id (NOT the entity_id) of the vCenter that corresponds to the vCenter id obtained in step # 4. This id string identifies the credentials stored for the root user for SSH connections to the VCSA. The password listed in the output will be the old, expired password. This needs to be updated.

  6. Update with the new password created directly on the VCSA in step # 1
    curl localhost/credentials-service/credentials/<id from Step # 5> -X PUT -H "Content-type:application/json" -d "<new password>" | json_pp
  7. Confirm that SDDC now lists the new password
    curl 'localhost/credentials-service/credentials?entityType=VCENTER&credentialType=SSH' | json_pp
  8. As the SDDC is powered on, it may have been trying to connect to the vCenter in the background, thereby locking the new password.
  9. If the new VCSA password is locked, power down the SDDC Manager appliance and follow KB Resetting root password in vCenter Server Appliance 6.5 / 6.7 / 7.x / 8.x to unlock the root account of vCenter.
  10. Power up the SDDC Manager appliance and confirm the UI now loads.

Additional Information

  • In rarer cases the UI will still not load if the default shell on the VCSA is the appliance shell.
    • Toggle the shell to the bash shell
      chsh -s /bin/bash root
  • In later versions of SDDC the API paths have changed and require authentication.
    • Generate a TOKEN on the SDDC Manager as root.
      TOKEN=$(curl -d '{"username" : "<sso username>", "password" : "<sso password>"}' -H "Content-Type: application/json" -X POST http://127.0.0.1/v1/tokens | jq -r '.accessToken')
    • Pull the vCenter root credentials.
      curl -X GET 'localhost/v1/system/credentials?entityType=VCENTER&credentialType=SSH' -H 'Content-Type: application/json' -H "Authorization: Bearer $TOKEN"
       
    • Put the new credentials into the SDDC DB.
      curl -X PUT 'localhost/v1/system/credentials/<id from Step # 5>' -d "<new password>" -H "Content-type:application/json" -H "Authorization: Bearer $TOKEN"| json_pp