SDDC Manager password remediation for vCenter root password fails with error: Failed executing command in VM with FQDN <vCenter_FQDN>, error : Failed to authenticate with the guest operating system using the supplied credentials.
search cancel

SDDC Manager password remediation for vCenter root password fails with error: Failed executing command in VM with FQDN <vCenter_FQDN>, error : Failed to authenticate with the guest operating system using the supplied credentials.

book

Article ID: 366978

calendar_today

Updated On:

Products

VMware SDDC Manager VMware Cloud Foundation 4.x

Issue/Introduction

  • SDDC Manager password remediation for vCenter root password fails below error in log file /var/log/vmware/vcf/operationsmanager/operationsmanager.log 

    com.vmware.vcf.passwordmanager.exception.PasswordUpdateException: Failed executing command in VM with FQDN <vCenter FQDN>, error : Failed to authenticate with the guest operating system using the supplied credentials. at com.vmware.vcf.passwordmanager.update.changers.VcenterRootUserChanger.doTest(VcenterRootUserChanger.java:76) ERROR [vcf_om,e35ece7b3eac706f,63e6] [c.v.v.p.u.c.VcenterRootUserChanger,om-exec-22] Exception in doTest of com.vmware.vcf.passwordmanager.update.changers.VcenterRootUserChanger
  • VC root password is matching with what is stored in SDDC Manager lookup_passwords and ssh connection to VC is working with the same credentials.
  • Below steps to create a test program on vCenter Server MOB using Guest Operation Manager doesn't return any PID
    • Turn on MOB: https://kb.vmware.com/s/article/2108405
    • Find out the VM MoId of vCenter Server VM using VC MOB:
      • Browse https://<vCenter FQDN>/mob/?moid=SearchIndex&method=findByDnsName
      • Enter below values, replace the <vCenter FQDN> with the FQDN of vCenter Server:
        datacenter = "" (leave it blank)
        dnsName = <vCenter FQDN>,
        vmSearch = true
      • Click Invoke Method and it will show the VM MoID.
    • Run a command using the Guest Operations Manager to confirm the PID:
      • Browse https://<vCenter FQDN>/mob/?moid=guestOperationsProcessManager&method=startProgram
      • Enter the following details to test the program execution (enter the VC VM MoId, username as root and the root password:

          vm =
            <vm type="VirtualMachine">VM_MoId_from_above_step</vm>


          auth =
           <auth xsi:type="internalvim25:NamePasswordAuthentication">

           <interactiveSession>false</interactiveSession>
           <username>root</username>
           <password>******</password>
           </auth>
          spec =
           <spec>

           <programPath>/bin/uname</programPath>
           <arguments>-a</arguments>
           </spec>
      • Click Invoke Method. Note down the Return value i.e. the process id. (e.g. 12345)
      • If the above steps fails to generate a PID, it indicates that there might be a problem with vCenter Server guest operations and steps in this KB will help to resolve the issue.

Resolution

  • Take snapshot of vCenter Server or take a backup
  • SSH to vCenter Server
  • Change directory to /etc/pam.d/

    cd /etc/pam.d/

  • Backup /etc/pam.d/vmtoolsd

    cp vmtoolsd vmtoolsd.pre_sshd_pre_mgmt

  • Replace /etc/pam.d/vmtoolsd with /etc/pam.d/sshd.pre_mgmt

    cp sshd.pre_mgmt vmtoolsd

  • Restart vgauth and vmtoolsd

    systemctl restart vgauthd vmtoolsd

  • Check the service status and confirm it is active/running

    systemctl --no-pager status vgauthd vmtoolsd

  • Retry VC root password remediation from SDDC Manager