Remediate Disconnected NSX and ALB Service Accounts in SDDC Manager (VCF 9.x)

Products

• VMware SDDC Manager / VCF Installer

Issue/Introduction

• NSX (svc-vcf-nsxmgr-####) or Avi (admin) accounts show a Disconnected status in SDDC Manager > Password Management.
• Remediation fails with the following error in /var/log/vmware/vcf/operationsmanager/operationsmanager.log: Unauthorized (com.vmware.vapi.std.errors.unauthorized) (statusCode:403)
• Error messages include: The credentials were incorrect or the account specified has been locked.
• VCF upgrades (e.g., 9.0.2 to 9.1.0) are blocked by the error: New update cannot be started while Domain mgmt-a or some of its components are not in active state.

Environment

• VCF 9.x

Cause

Authentication failure occurs due to password desynchronization between the SDDC Manager database and the component APIs (NSX/Avi). This is frequently caused by:

1. Out-of-band password changes performed directly on the appliance.
2. Account lockouts triggered by expired credentials or excessive failed login attempts from the SDDC Manager orchestration layer.

Resolution

Perform the following steps to synchronize credentials.

Part 1: Remediate NSX Service Accounts

1. Log in to the NSX Manager via SSH as root.
2. Stop the API service:
   bash

    

   /etc/init.d/nsx-mp-api-server stop

3. Clear the password history to allow reuse of the SDDC-stored password:
   bash

    

   echo "" > /etc/security/opasswd

4. Set the password to match the value currently stored in the SDDC Manager database (refer to KB 327195 for retrieval steps):
   bash

    

   passwd <service_account_name>

5. Create a flag file to trigger a cluster credential reset:
   bash

    

   touch /var/vmware/nsx/reset_cluster_credentials

6. Restart the API service:
   bash

    

   /etc/init.d/nsx-mp-api-server start

Part 2: Remediate Avi (ALB) Admin Accounts

1. SSH to the Avi Controller IP.
2. Enter the shell:
   bash

    

   cliattach controller <Controller_IP>

3. Switch to root and navigate to the scripts directory:
   bash

    

   sudo -icd /opt/avi/scripts

4. Run the password reset script and follow prompts to set the password to match SDDC Manager:
   bash

    

   python3 admin_pwd_reset.py

Part 3: Finalize in SDDC Manager

1. Navigate to SDDC Manager > Password Management.
2. Locate the affected accounts and click Remediate.
3. Verify the status changes to Active (Green).

Additional Information

"New update cannot be started while Domain xxxxx or some of its components are not in active state" - error encountered when attempting VCF 9.0.2 to 9.1.0 upgrade of SDDC Manager 

Retrieve the service accounts credentials from SDDC Manager 

How to reset AVI Controller's admin password 

NSX service account is disconnected in SDDC manager Password Management