Host Commissioning Fails in SDDC Manager Due to Stale ESXi Service Accounts.
search cancel

Host Commissioning Fails in SDDC Manager Due to Stale ESXi Service Accounts.

book

Article ID: 451550

calendar_today

Updated On:

Products

VMware vSAN

Issue/Introduction

Symptoms:

  • Commissioning the hosts which had already gone through multiple commission and decommission cycles on the SDDC Manager fails at the "Create New Local ESX Host User" stage.

    All hosts are skipped, failing the workflow



  • Configuration status displays as "Error". 



Environment

  • VCF 9.x
  • SDDC Manager 9.x

Cause

  • During commissioning, SDDC Manager attempts to create a dedicated local service account (svc-####) on each target ESXi host.
  • Incomplete decommissioning from previous cycles leaves stale service accounts on the host. When SDDC Manager attempts to validate access using newly generated credentials, authentication fails due to a credential mismatch with the existing stale user.
  • The /var/log/vmware/vcf/operationsmanager/operationsmanager.log file confirms that the SDDC Manager is attempting to create a service account on the ESXi host during the commissioning workflow. However, it detected that the user already exists and the user name and the password for the existing svc user is incorrect. 

    2026-07-29T09:56:59.567+0000 INFO  [vcf_om,################################,b4b1] [c.v.e.s.v.contract.CreateEsxiUser,pool-2-thread-19] Creating local user svc-###-###-###-###-### in ESXi host ###-####-####-##.####.#####
    2026-07-29T09:56:59.630+0000 DEBUG [vcf_om,################################,b4b1] [c.v.e.s.c.c.v.vsphere.VcManagerBase,pool-2-thread-19] Existing ESXi users on ###-####-####-##.####.##### are: [root, dcui, vpxuser, svc-###-###-###-###-###]
    2026-07-29T09:56:59.630+0000 WARN  [vcf_om,################################,b4b1] [c.v.e.s.v.contract.CreateEsxiUser,pool-2-thread-19] User svc-###-###-###-###-### already exists
    2026-07-29T09:56:59.630+0000 INFO  [vcf_om,################################,b4b1] [c.v.e.s.v.contract.CreateEsxiUser,pool-2-thread-19] Checking if user svc-###-###-###-###-### is operational with given credentials
    2026-07-29T09:57:03.705+0000 ERROR [vcf_om,################################,b4b1] [c.v.e.s.c.c.v.vsphere.VsphereClient,pool-2-thread-19] Unable to connect to svc-###-###-###-###-###@###-####-####-##.####.#####
    com.vmware.vim.binding.vim.fault.InvalidLogin: Cannot complete login due to an incorrect user name or password.
    2026-07-29T09:57:17.245+0000 ERROR [vcf_om,################################,4814] [c.v.e.s.c.c.v.vsphere.VcManagerBase,pool-2-thread-8] Cannot complete login due to incorrect credentials: ###-####-####-##.####.#####, svc-###-###-###-###-###.
    2026-07-29T09:57:17.245+0000 ERROR [vcf_om,################################,4814] [c.v.e.s.v.contract.CreateEsxiUser,pool-2-thread-8] Unable to login to ###-####-####-##.####.##### as svc-###-###-###-###-### user: Cannot complete login due to an incorrect user name or password.
    com.vmware.vim.binding.vim.fault.InvalidLogin: Cannot complete login due to an incorrect user name or password.
    2026-07-29T09:57:03.716+0000 ERROR [vcf_om,################################,b4b1] [c.v.e.s.c.c.v.vsphere.VsphereClient,pool-2-thread-19] Failed to connect to https://###-####-####-##.####.#####:443/sdk as svc-###-###-###-###-###
    com.vmware.vim.binding.vim.fault.InvalidLogin: Cannot complete login due to an incorrect user name or password.

  • The /var/run/log/hostd.log file on the affected host confirms that host PAM authentication rejected the login attempt.

    2026-07-29T09:56:59.686Z -ERROR Hostd ####### [esx@#### sub="Default" opID="c8d9636e" rhost="##.#.##.#" sid="########"] [module:pam_lsass]pam_do_authenticate: error [login:svc-###-###-###-###-###][error code:2]
    2026-07-29T09:56:59.686Z -ERROR Hostd ####### [esx@#### sub="Default" opID="c8d9636e" rhost="##.#.##.#" sid="########"] [module:pam_lsass]pam_sm_authenticate: failed [error code:2]
    2026-07-29T09:56:59.688Z -WARNING Hostd ####### [esx@#### sub="Vimsvc.HaSessionManager" opID="c8d9636e" sid="########"] Rejected password for user svc-###-###-###-###-### from ##.#.##.# - session=########-####-####-####-############
    2026-07-29T09:56:59.688Z -INFO Hostd ####### [esx@#### sub="Vimsvc.ha-eventmgr" opID="c8d9636e" sid="########"] Event 4119 : Cannot login svc-###-###-###-###-###@##.#.##.#

  • Running the command "esxcli system account list" on the affected host confirms an existing stale svc account.

    esxcli system account list
    User ID                   Description                                  Shell access  Last password change
    ------------------------  -------------------------------------------  ------------  --------------------
    svc-###-###-###-###-###   ESXi User                                            true  2026-07-26

Resolution

  1. Log into the affected ESXi host via SSH using root credentials.
  2. Remove the stale service account:

    esxcli system account remove -i svc-###-###-###-###-###

  3. Retrigger the host commissioning workflow in SDDC Manager.