NSX alarm - "Node local user account with username svc-sddc_manager_hostname-nsx_hostname-#### is temporarily locked for VCF Networking web access" - VCF 9.1.1

Products

• VMware SDDC Manager / VCF Installer

Issue/Introduction

• An NSX alarm is triggering periodically:

Node local user account with username svc-sddc_manager_hostname-nsx_hostname-#### is temporarily locked for VCF Networking web access, due to 5 failed authentication attempts were made over this account in last 900 seconds. Account will be locked for 900 seconds, during this any attempt to authenticate user svc-sddc_manager_hostname-nsx_hostname-#### for UI or API access will be denied, due to lockout policy is in effect on the target user account. After 900 seconds account will be automatically unlocked and user svc-sddc_manager_hostname-nsx_hostname-#### will again be allowed to web access VCF Networking via UI or API. Refer KB to understand more details on lockout policy and its impact.

• Stale locks have been identified on the SDDC Manager, having ssh'd onto it as root, su and then running this command below: /usr/pgsql/13/bin/psql -h localhost -U postgres -d platform -c "select * from lock;"

Found active workflow for OPERATIONS_MANAGER, Description: Resource of type NSX locked by service (ID: OPERATIONS_MANAGER) and operation (ID: ########-####-####-####-############)

Environment

VCF 9.1.1

Cause

Stale task entries residing in the PostgreSQL platform database's lock table are improperly holding 'EXCLUSIVE' and 'LIMITED' locks on SDDC Manager resources, preventing new workflows (including NSX password management ones) from proceeding - resulting in password synchronization issues manifested as account lockouts on the NSX Manager.

Resolution

Implement these steps:

1. Take a snapshot of the SDDC Manager virtual machine.

2. Log in to the SDDC Manager via SSH as vcf, then su.

3. Identify the stale locks:

 /usr/pgsql/13/bin/psql -h localhost -U postgres -d platform -c "select * from lock;"

4. Delete each stale lock entry, replacing <id_UUID> with the specific ID of each stale task found in the database:

 /usr/pgsql/13/bin/psql -h localhost -U postgres -d platform -c "delete from lock where id='<id_UUID>';"

5. Re-run the 'select * from lock;' query again to verify that the lock table is clear of the stale entries.

6. The NSX Manager alarm is expected to no longer be triggered, given that the lock that was blocking password synchronization is removed.