NSX-T Manager user accounts disconnected in SDDC Manager password management

Products

• VMware Cloud Foundation
• VMware NSX

Issue/Introduction

• NSX manager user accounts such as root, admin, and audit users, in SDDC manager password management are showing as disconnected.
• When trying to remediate the password for each user, some may fail to be remediated and the user account remained disconnected. 
• When trying to login locally to NSX UI (https:// nsx-manager/login.jsp?local=true) with the last known credential from SDDC manager, a generic error message is displayed indicating the password is incorrect or user has been locked-out. 

 

Environment

VMware Cloud Foundation

VMware NSX

Cause

Possible reasons for user account disconnection:

• NSX user account password expired
• NSX user account locked out due to too many failed logins
• NSX user account password was manually updated from NSX and not rotated through SDDC manager
• SDDC manager does not have the correct NSX user passwords

Resolution

 To resolve the issue in NSX manager appliance, check the following options:

• Can log in as admin from the UI and all user passwords are working

  • Check user password expiration status: nsxcli> get user <local-user-name> password-expiration
  • If the user password is expired, this can be reset or simply disabling password-expiration can be used therefore the password no longer expires.
    • To reset user password when current password is known: set user <local-user-name> password 
    • To disable password expiration for a user: clear user <local-user-name> password-expiration
  • Verified user password no longer expired: get user <local-user-name> password-expiration
  • Test if local user can login through NSX UI or a SSH session.
  • Use "Remediate" option in SDDC password management page for each of the NSX local user account that has the password updated or cleared of password-expiration.

• Cannot log in as a local user on NSX UI or SSH, however, root password is still working

  • To reset any local user's password when root user password is working, please follow the first half of this Techdoc. 
  • Test if local user can login through NSX UI or a SSH session.
  • Use "Remediate" option in SDDC password management page for each of the NSX local user account with reset password.

•  Root user password is lost or incorrect

  • Admin user can reset a user's password only when the current password is known, otherwise, root user access is required to reset any user's passwords.
  • To reset root user password, please follow the bottom half of the instruction in this Techdoc.
  • After the root user password recovery, follow the first half of the instruction of the same Techdoc to reset all other local user passwords if needed.
  • Use "Remediate" option in SDDC password management page for each of the NSX local user account with reset password.

•  Root and admin user passwords are already updated, however, remediate passwords in SDDC manager still fails and admin user cannot login through NSX UI or a SSH session

  • The affected users have their passwords updated using above methods.
  • Admin user still cannot login to NSX UI using the reset credential, the error message indicates incorrect password or user is locked-out.
  • Root user can login through NSX manager VM console but not through a SSH session (if permitRootLogin is enabled for SSH).
  • Option 1, disable user lock-out policy
    • Use a NSX manager VM console log in as root user
    • root#: su admin
    • nsxcli> set auth-policy api lockout-period 0
    • nsxcli> set auth-policy cli lockout-period 0 
    • Verify local users can log in from SSH or NSX UI again
    • Use "Remediate" option in SDDC password management page for each of the NSX local user account with reset password.
    • Re-enable lock-out policy to default value: 
      • nsxcli> set auth-policy api lockout-period 900
      • nsxcli> set auth-policy cli lockout-period 900
  • Option 2, rolling reboot of NSX managers
    • After all NSX manager nodes are rebooted one at a time, confirm local users can log in through NSX UI or a SSH session
    • Use "Remediate" option in SDDC password management page for each of the NSX local user account with reset password.

Additional Information

Impact/Risks:
NONE - The steps provided are safe to run with no downtime on any of the components involved.