Admin password rotation failed in SDDC Manager, and inventory sync and manually setting a new password does not work

Products

• VCF Operations

Issue/Introduction

SDDC password rotation failed, customer followed guidelines to reset the admin password. However the admin password is getting locked immediately, even after running the standard unlock and sync commands

$VMWARE_PYTHON_BIN $VCOPS_BASE/../vmware-vcopssuite/utilities/sliceConfiguration/bin/vcopsSetAdminPassword.py --unlock

$VMWARE_PYTHON_BIN $VCOPS_BASE/../vmware-vcopssuite/utilities/sliceConfiguration/bin/vcopsSetAdminPassword.py --sync

Environment

Aria Operations 8.18x

VMware Cloud Foundation 5.2x

Aria LCM 8.18x

Cause

While explicit root cause errors were not immediately identifiable in standard logs, rapid and immediate account lockouts typically occur when integrated management systems repeatedly attempt to authenticate using stale credentials. To trace the behavior, the following command was used to monitor real-time log modifications during a password rotation attempt:

watch -n5 -d "date; find /var/log -newermt '5 seconds ago'"

Resolution

Executed the following sequence of steps to successfully rotate the admin user password and prevent integrated systems from locking the account:

1. Set the new password locally using the Aria Operations command line:
$VMWARE_PYTHON_BIN $VCOPS_BASE/../vmware-vcopssuite/utilities/sliceConfiguration/bin/vcopsSetAdminPassword.py --reset

2. Remediated and updated the password in SDDC Manager.

3. Verified the new password via successful login in the Aria Operations UI.

4. Performed an inventory sync in Aria Suite Lifecycle (LCM).

5. Verified the successful sync in the LCM UI.

6. Ran a final verification in SDDC Manager using the lookup_passwords utility.

Additional Information

Customer had implemented the following work arounds with no success:

How to Reset the Admin Password in VMware Aria Operations / VCF Operations

How to Reset the Root Password for VCF Operations

Log in to Admin and Product UI as admin fails in Aria Operations