Re-create missing SDDC Manager Service account on an ESXi host

Products

• VMware Cloud Foundation
• VMware vSphere ESXi

Issue/Introduction

• The purpose of this KB is to provide the steps to re-create the missing service account on an ESXi host, if it already exists on the SDDC Manager.
   

Note: If the service account does not exist on the SDDC Manager as well, please use the following KB instead: Missing ESXi service accounts in SDDC Manager after Bringup

• Missing the SDDC Manager created service account on the ESXi hosts.
• Unable to perform any administrative tasks against the ESXi host via the SDDC Manager
• Config drift upgrade will fail at vSAN LSOM update stage on certain hosts.
  • LCM logs will report:

    Failed to update vSAN LSOM settings on the host esxi01.example.com
    Failed to connect to esxi01.example.com

Environment

VMware Cloud Foundation 4.x

Cause

If the host has been rebuilt/reconfigured and then re-added to the vCenter without going through the decommission / commission process from the SDDC Manager, we may be missing the service account on the ESXi host.

Resolution

1. Check if the service account is missing. Run the following command in an SSH session on the affected ESXi host:

   esxcli system account list

   Sample output

   [root@esxi01:~] esxcli system account list
   User ID         Description                                Shell access
   --------------  -----------------------------------------  ------------
   root            Administrator                                      true
   dcui            DCUI User                                          true
   vpxuser         VMware Workstation administration account          true
   cloudadmin      ESXi User                                          true
   lldpVim-user    ESXi User                                          true
   mux_user        ESXi User                                          true
   da-user         ESXi User                                          true
   nsx-user        ESXi User                                          true
   

2. Since the svc-vcf-esxi01 account is missing, we will need to manually re-create the service account on the ESXi host.
   
   • The account will also be missing from the /etc/passwd and /etc/shadow files
   
   
   1. Get an SSH session to the affected ESXi host(s)
   2. Create the service account:

      esxcli system account add --id svc-vcf-esxi01 --password <password> --password-confirmation <password>

   3. Add the required permission:

      esxcli system permission set --id svc-vcf-<esxi_shortname> --role Admin

   4. Permission for svc-vcf-esxi01 should look like this:

      [root@esxi01:~] esxcli system permission list
      Principal       Is Group  Role      Role Description
      --------------  --------  --------  ----------------
      svc-vcf-esxi01     false  Admin     Full access rights

3. Login to the SDDC Manager UI, and go to the Password Manager
4. Locate the svc-vcf service account for the host in question and REMEDIATE the password, using the password supplied while creating the account above.
5. (Optional) Rotate the password for the svc-vcf account to confirm that SDDC Manager is able to interact with the host.