Troubleshooting VMware SASE Orchestrator and Gateway SSH Login Issues
Article ID: 312371
Updated On:
Products
VMware SD-WAN by VeloCloud
Issue/Introduction
This KB article covers what an Operator user with a vcadmin user account must do to troubleshoot an inability to SSH login into either an Orchestrator or Gateway because their password has expired.
Symptoms:
An Operator with a vcadmin account attempts to SSH login into a VMware SASE (also SD-WAN) Orchestrator or Gateway using a 4.x or later build and which is deployed via cloud-init and has SSH password authentication only. The result of the SSH login attempt is an error that reads:
"WARNING: Your password has expired.
You must change your password now and login again!
Connection to x.x.x.x closed"
Note: In the event that the user account has expired the error would read:
"Your account has expired; please contact your system administrator."
In 3.x versions, the user would be prompted to reset their password but on 4.x Orchestrators and Gateways a user only observes an error and no explicit way of resolving it.
Symptoms:
An Operator with a vcadmin account attempts to SSH login into a VMware SASE (also SD-WAN) Orchestrator or Gateway using a 4.x or later build and which is deployed via cloud-init and has SSH password authentication only. The result of the SSH login attempt is an error that reads:
"WARNING: Your password has expired.
You must change your password now and login again!
Connection to x.x.x.x closed"
Note: In the event that the user account has expired the error would read:
"Your account has expired; please contact your system administrator."
In 3.x versions, the user would be prompted to reset their password but on 4.x Orchestrators and Gateways a user only observes an error and no explicit way of resolving it.
Environment
VMware SD-WAN by VeloCloud
Cause
A VMware SASE Orchestrator and Gateway using 4.0.0 or later software, the system administrative account vcadmin password expiration policy is set by default for 90 days.
If the vcadmin password is not reset within the 90 day window, upon login the vcadmin user is notified of their password expiration and the SSH session is terminated without an option to reset the password.
If the vcadmin password is not reset within the 90 day window, upon login the vcadmin user is notified of their password expiration and the SSH session is terminated without an option to reset the password.
Resolution
WARNING: Because this involves rebooting the operating system (OS) of the Orchestrator or Gateway with the expired password, it is recommended to follow these troubleshooting steps during a maintenance window. If done outside of a maintenance window, there should be an expectation of a brief customer traffic disruption on a Gateway or a brief pause in configuration updates on an Orchestrator during the OS reboot.
The following steps describe the procedure to reset the vcadmin password via the recovery OS mode on the Orchestrators or Gateways deployed on the ESXi hypervisor:
7. Once the text has been added, select CTRL + x on your keyboard to continue booting with bash path into a root shell with no password.
8. From this GRUB menu select (recovery mode) and press Enter.
9. Wait until the OS is loaded in the recovery mode and when prompted, press Enter for Maintenance. You will be presented with the root shell prompt.
10. To change the vcadmin password, type 'passwd vcadmin' and provide a new secure password. (This is equally true for any user name as in the below screenshot where the command 'passwd root' is shown for a root user account). The new password must contain at least 1 uppercase letter, 1 digit, 1 non-alphanumeric character, and be 14 characters or longer.
11. Once the password is changed, verify password information expiry dates with command 'chage -l vcadmin'
12. Reboot the OS by typing 'reboot' in the shell and press enter.
Not Recommended: If you desire to change the maximum number of days between password changes, you can utilize the command 'chage -M 120 vcadmin' where 120 is the number of days between password change.
Workaround:
A recommended best practice is to always configure SSH key authentication versus just SSH password authentication only. This has the added benefit of preventing Orchestrator and Gateway user login issues related to expired passwords or user accounts.
The following steps describe the procedure to reset the vcadmin password via the recovery OS mode on the Orchestrators or Gateways deployed on the ESXi hypervisor:
1. Login to the ESXi host user interface and locate the Orchestrator or Gateway VM.
2. Right click the running Orchestrator VM and from the Console sub-menu select 'Open Browser Console'.
3. From the 'Console Actions' menu select 'Guest OS, Restart Guest OS' to gracefully restart the Orchestrator software.
4. Ensure that the mouse focus is in the console window at all times and once the 'restarting' message appears, press down and hold the SHIFT button until the GRUB Menu is loaded.
5. You will then see a screen like the one below. Highlight Option 2 (recovery mode), but DO NOT press the Enter key on your keyboard. Instead press the 'e' key on your keyboard which allows you to edit commands.
6. Once the 'e' key is pressed, you will see the screen shown below which is now in Edit mode.
At the line that begins linux, add the text rw init=/bin/bash to the end of the line as shown below:
2. Right click the running Orchestrator VM and from the Console sub-menu select 'Open Browser Console'.
3. From the 'Console Actions' menu select 'Guest OS, Restart Guest OS' to gracefully restart the Orchestrator software.
4. Ensure that the mouse focus is in the console window at all times and once the 'restarting' message appears, press down and hold the SHIFT button until the GRUB Menu is loaded.
5. You will then see a screen like the one below. Highlight Option 2 (recovery mode), but DO NOT press the Enter key on your keyboard. Instead press the 'e' key on your keyboard which allows you to edit commands.
6. Once the 'e' key is pressed, you will see the screen shown below which is now in Edit mode.
At the line that begins linux, add the text rw init=/bin/bash to the end of the line as shown below:
7. Once the text has been added, select CTRL + x on your keyboard to continue booting with bash path into a root shell with no password.
8. From this GRUB menu select (recovery mode) and press Enter.
9. Wait until the OS is loaded in the recovery mode and when prompted, press Enter for Maintenance. You will be presented with the root shell prompt.
10. To change the vcadmin password, type 'passwd vcadmin' and provide a new secure password. (This is equally true for any user name as in the below screenshot where the command 'passwd root' is shown for a root user account). The new password must contain at least 1 uppercase letter, 1 digit, 1 non-alphanumeric character, and be 14 characters or longer.
11. Once the password is changed, verify password information expiry dates with command 'chage -l vcadmin'
12. Reboot the OS by typing 'reboot' in the shell and press enter.
Workaround:
A recommended best practice is to always configure SSH key authentication versus just SSH password authentication only. This has the added benefit of preventing Orchestrator and Gateway user login issues related to expired passwords or user accounts.
Feedback
Yes
No
Powered by
