Unable to change the password with old password or a new password for a user - "Authentication token manipulation Error. Password unchanged" in VMware Cloud Foundation

Products

• VMware Cloud Foundation

Issue/Introduction

• Unable to change the password for the user vcf or root on the appliance. 
• Avoid failure of any task on SDDC related to the username root and vcf of SDDC 

Symptoms:
Unable to change a password for user root / vcf if any of it got expired . Even if a brand new password has been provided which is not used in the past gets rejected 

Environment

VMware Cloud Foundation 4.x

Cause

The file "system-password"   under /etc/pam.d is corrupted or any modification has been done . Also the order of the file matters . Example ,

 

Similar File structure that do not allow us to change password 

> cat /etc/pam.d/system-password
# Begin /etc/pam.d/system-password
  
# use sha512 hash for encryption, use shadow, and try to use any previously
# defined authentication token (chosen password) set by any prior module
password requisite pam_cracklib.so dcredit=-1 ucredit=-1 lcredit=-1 ocredit=-1 minlen=8 minclass=4 difok=4 retry=3 maxsequence=0 enforce_for_root
password  required    pam_unix.so       sha512 shadow try_first_pass

password required pam_pwhistory.so enforce_for_root use_authtok remember=5 retry=3

# End /etc/pam.d/system-password

 

Similar File Structure that Allow us to change password 

❯ cat /etc/pam.d/system-password
# Begin /etc/pam.d/system-password

# use sha512 hash for encryption, use shadow, and try to use any previously
# defined authentication token (chosen password) set by any prior module
password  requisite   pam_cracklib.so
password  required    pam_pwhistory.so enforce_for_root  remember=5 retry=3
password  required    pam_unix.so       sha512 shadow try_first_pass

# End /etc/pam.d/system-password

 

Resolution

Ensure you have taken a snapshot of the SDDC Manager VM before making any changes to the configuration files.

• Copy the file system-password to any location to have a backup of it  
  • cp /etc/pam.d/system-password ~
• Replace the content of the file system-passwordr as system-password
  • cp /etc/pam.d/system-passwordr  /etc/pam.d/system-password
• If we need to use the same password that we used before 
  • Clear the content of /etc/security/opasswd file -  echo "" > /etc/security/opasswd
• Now change the password using 
  • passwd <username>

NOTE: There will be two file /etc/pam.d/system-password and /etc/pam.d/system-passwordr watch out for the "r"

Additional Information

Impact/Risks:
Make sure we have a backup of the file /etc/pam.d/system-password or a snapshot of SDDC to avoid any confusion with the file names ( /etc/pam.d/system-password and /etc/pam.d/system-passwordr)