vmware-system-user password reset: VCF Services Runtime and VCF Automation

Products

• VCF Automation
• VCF Operations
• VMware Cloud Foundation

Issue/Introduction

• Password reset procedure for vmware-system-user local Linux account for VCF Services Runtime 9.1 and VCF Automation 9.1 (VCFA)
• You are unable to login to the VCF Services Runtime or VCFA virtual machines using either the Virtual Console or Secure Shell (SSH) with the vmware-system-user account.
  

Environment

• VCF Management Services 9.1.0, 9.1.1
• VCF Services Runtime 9.1.0, 9.1.1
• VCF: 9.1.1

Resolution

Note:

• The below process should only be followed when you are unable to manage the vmware-system-user password via the Fleet Management UI/API.
• If the password is known and you simply wish to reset, see Update Passwords for VMware Cloud Foundation Components.

The procedure requires multiple steps:

1. Identify one of the VCF services runtime control plane node virtual machines
   1. Log in to the VCF Operations interface at https://<vcf_operations_fqdn> as a user assigned the Administrator role.
   2. Navigate to Build > Lifecycle > VCF Management.
   3. On the Components tab, select the VCF services runtime or VCF Automation, depending on which product you need to reset the vmware-system-user password for.
   4. Scroll to the Nodes section.
   5. Identify a Control Pane node and note the IP address and VM name
2. Reboot the virtual machine into single user mode
   1. Navigate to the vSphere Client where the cluster is deployed and find the virtual machine. Open a virtual console.
   2. Right click on the virtual machine, choose Power, then Restart Guest OS.
   3. Switch back to the virtual console and once you see the Photon bootloader screen, press e.
   4. Use the cursor keys to position the cursor at the end of the kernel command line and add rw init=/bin/bash to the end, then press F10 to continue the boot procedure.
3. Create a temporary user and set its password (15 characters, no more than 3 sequential characters of same class)

   useradd temp
   
   passwd temp

4. Give the new user sudo privileges

   echo "temp ALL=(ALL) ALL" >> /etc/sudoers.d/90-cloud-init-users

5. Reboot the VM

   reboot -f

6. Login via an ssh session using the newly created temp user
7. Switch to Root

   sudo su -

8. Update the password in the Kubernetes secret by running the following commands:

   CLUSTER=$(kubectl get configmap global-config -n vmsp-platform -o jsonpath='{.data.cluster\.name}')
    
   read -sp "Enter password: " NEW_PASSWORD
   
   export NEW_PASSWORD 
   
   HASH=$(printf '%s' "${NEW_PASSWORD}" | vmsp passwd --password-stdin | tr -d '\n' | base64 -w0)
   
   kubectl patch secret "${CLUSTER}-ssh-password-secret" -n vmsp-platform --type=merge -p "{\"data\":{\"sshPassword\":\"${HASH}\"}}"

9. Confirm the vmware-system-user can now login
10. Remove the temporary user

    sudo userdel temp
    

    Note: Step 8 will immediately trigger all the virtual machines in the cluster to update the password for vmware-system-user.

Additional Information

• Resetting Passwords for VMware Cloud Foundation Services Runtime