SDDC Manager fails to remediate NSX credentials after successful manual reset on NSX appliances

Products

• VMware NSX

Issue/Introduction

This article provides troubleshooting guidance for a scenario where NSX Manager root, admin, or audit passwords have been successfully reset at the appliance level to match the SDDC Manager database, but the SDDC Manager UI continues to report a Disconnected status or fails remediation tasks.

 

• NSX local accounts (root, admin, or audit) are confirmed to be accessible via direct SSH across all NSX Manager nodes.

• Manual verification via the command line interface (faillock and chage -l) confirms that the accounts are not locked and password expiration flags are cleared.

• In the SDDC Manager UI, the NSX component status remains Disconnected under Security > Password Management.

• Remediation attempts in SDDC Manager fail with the following UI error: "The credentials were incorrect or the account specified has been locked."

• The following error is observed in the SDDC Manager logs (/var/log/vmware/vcf/operationsmanager/operationsmanager.log): ERROR [vcf_om] [c.v.v.p.u.c.NsxtManagerSshChanger] The credential test failed for audit@<NSX-FQDN> due to com.vmware.vcf.passwordmanager.exception.Fault: Failed executing command in VM.

 

Environment

VMware Cloud Foundation (VCF) 5.x, 9.x

VMware NSX 4.x

 

Cause

This issue occurs due to a synchronization failure in the SDDC Manager orchestration layer, often triggered by underlying OS-level password aging policies on the NSX appliances.

Even if the password is manually corrected and valid, the underlying Linux OS may flag the account to force a password change upon the next login. Because SDDC Manager uses automated, non-interactive SSH scripts to validate credentials, it cannot bypass or respond to this forced-change interactive prompt, resulting in a failed credential test. Alternatively, the SDDC Manager operationsmanager service may simply be stuck in a cached failure state.

Resolution

Before proceeding, ensure you have followed the primary appliance-level recovery steps outlined in KB 314657 to clear opasswd history and faillock lockouts.

Step 1: Clear OS-Level Forced-Change Prompts Ensure the NSX appliance OS is not forcing a password change on the next login, which blocks automated SSH scripts.

1. SSH into each NSX Manager node in the cluster as root.

2. Run the following command to update the last password change date to today, clearing the prompt:

 chage -d $(date +%Y-%m-%d) audit

(Note: Replace audit with root or admin if you are troubleshooting a different account).

Step 2: Restart SDDC Management Services If the backend is healthy and the chage prompts are cleared but the UI persists in a failed state, restart the Operations Manager service to clear cached database states.

1. SSH into the SDDC Manager appliance as vcf and su to root.

2. Restart the service:

   systemctl restart operationsmanager
   

Step 3: Retry Remediation

1. Log in to the SDDC Manager UI.

2. Navigate to Security > Password Management.

3. Select the affected account and click Remediate.

Additional Information

For the prerequisite steps on clearing NSX appliance password history and account lockouts, refer to: Credential operations fail on NSX components in VMware Cloud Foundation (KB 314657)