SDDC Manager password rotate for Backup user fails with error : Failed executing command in VM <vm_name>, error: Failed to authenticate with the guest operation system using the supplied credentials

Products

• VMware Cloud Foundation
• VMware SDDC Manager

Issue/Introduction

• Password rotation for SDDC Manager backup user fails with error

  Failed executing command in VM <vm_name>, error: Failed to authenticate with the guest operation system using the supplied credentials

• Resetting password expiry for SDDC Manager Backup user

You will find log entries reported by operationsmanager similar to:

/var/log/vmware/vcf/operationsmanager/operationsmanager.log

YYYY-MM-DDTHH:MIN:SEC INFO [vcf_om,67d8c5cd7d624df7ba7995f3682874fc,335e] [c.v.e.s.c.c.v.vsphere.VsphereClient,om-exec-11] Successfully logged in to https://vCenter FQDN:443/sdk
YYYY-MM-DDTHH:MIN:SEC DEBUG [vcf_om,67d8c5cd7d624df7ba7995f3682874fc,335e] [c.v.e.s.c.c.v.vsphere.VcManagerBase,om-exec-11] Finding vms with host name <Backup FQDN>
YYYY-MM-DDTHH:MIN:SEC DEBUG [vcf_om,67d8c5cd7d624df7ba7995f3682874fc,335e] [c.v.v.p.helper.GuestProgramService,om-exec-11] 1 VMs found with FQDN: <Backup FQDN>
YYYY-MM-DDTHH:MIN:SEC DEBUG [vcf_om,67d8c5cd7d624df7ba7995f3682874fc,335e] [c.v.v.p.helper.GuestProgramService,om-exec-11] 1 powered on VMs found with FQDN: <Backup FQDN>
YYYY-MM-DDTHH:MIN:SEC DEBUG [vcf_om,67d8c5cd7d624df7ba7995f3682874fc,335e] [c.v.v.p.helper.GuestProgramService,om-exec-11] Starting guest program '/bin/uname' on <Backup FQDN>
YYYY-MM-DDTHH:MIN:SEC ERROR [vcf_om,67d8c5cd7d624df7ba7995f3682874fc,335e] [c.v.v.p.helper.GuestProgramService,om-exec-11] Failed to authenticate with the guest operating system using the supplied credentials.
YYYY-MM-DDTHH:MIN:SEC ERROR [vcf_om,67d8c5cd7d624df7ba7995f3682874fc,335e] [c.v.v.p.u.c.VCenterToolsChanger,om-exec-11] Exception in doTest of com.vmware.vcf.passwordmanager.update.changers.BackupUserChanger
com.vmware.vcf.passwordmanager.exception.Fault: Failed executing command in VM with FQDN , error : Failed to authenticate with the guest operating system using the supplied credentials.
Caused by: com.vmware.vim.binding.vim.fault.InvalidGuestLogin: Failed to authenticate with the guest operating system using the supplied credentials.

 

Environment

VMware Cloud Foundation

Cause

SDDC Manager backup user's password expired

Resolution

1. Take snapshot of SDDC Manager VM
2. SSH to SDDC Manager with vcf account and su to root
3. Run the below commands to check the backup user status

   id backup

   Sample Output

   root@vcf [ /home/vcf ]# id backup
   uid=1001(backup) gid=993(backup) groups=993(backup)

   
   
   

   chage -l backup

   Sample output

   root@vcf [ /home/vcf ]# chage -l backup
   
   Last password change : Apr 21, 2021
   Password expires : Apr 21, 2022 
   Password inactive : never
   Account expires : never
   Minimum number of days between password change : 0
   Maximum number of days between password change : 365
   Number of days of warning before password expires : 7

   Password expires : Apr 21, 2022 > If you see here , it looks like the password was already expired

4. Run the below command to change the password expiry for backup user

   chage -I -1 -m 0 -M 99999 -E -1 backup

   This command will change the make password expiry to never and set the minimum and maximum number of days between password change from 0 to 99999 so that we need not change the backup user account password every 365 days once.
5. Confirm the password expiry status

   chage -l backup

   
   Sample output

   Last password change : Apr 21, 2021
   Password expires : never
   Password inactive : never
   Account expires : never
   Minimum number of days between password change : 0
   Maximum number of days between password change : 99999
   Number of days of warning before password expires : 7
   Retry the password rotation for backup user

6. Retry the password rotation for backup user