Getting error “Password management operation failed in pre-validation Stage” when performing password management task using SDDC manager.

Products

• VMware SDDC Manager

Issue/Introduction

When a user is performing password management tasks such as Rotate/Update/Remediate, the operation fails with one of the following errors.

• “Password management operation failed in pre-validation Stage”.
• "Password management operation failed".

Task "Health-Check operation for..." with Subtask of "Pre-validation [Validate Source, Validate Disk Space, Verify Inventory..." shows status "Failed".

Environment

VCF 5.x

Cause

Stale status entries in the SDDC database causes the Password Management workflows to fail.

Resolution

If account passwords for nodes have expired and showing as disconnected in SDDC manager follow the steps within KB Root user account is disconnected in SDDC Manager password management and wait some time for syncing to complete. Proceed with attempting to cancel the current task. If the cancellation is successful, the banner showing the "Retry or Cancel" options should disappear. This will then allow you to run the password task.

If you encounter this issue and the steps above do not resolve the issue please follow the steps below.

 

Note: Before you proceed further, please take a snapshot of the SDDC manager VM.

Method 1:

• Login to SDDC manager via SSH session as VCF user.
• Switch to root user using 'su' command.
  • Connect to database: psql -h localhost -U postgres
  • Navigate to operations manager database: \c operationsmanager
  • List out the prevalidation failed tasks for password management: SELECT workflow_id, operation_type, transaction_status FROM passwordmanager.password_operations WHERE transaction_status='FAILED' OR transaction_status='PREVALIDATION_FAILED';
  • Delete old prevalidation failed task from the database: UPDATE passwordmanager.password_operations SET transaction_status='USER_CANCELLED' WHERE transaction_status='FAILED' OR transaction_status='PREVALIDATION_FAILED';
• Retry password management task from SDDC manager web UI.

 

Method 2:

If the issue persists, review 'DB_Dump' logs from offline bundle to find entries similar to "COPY public.nsxt_edge_cluster (id, creation_time, modification_time, status, name, nsxt_edge_nodes, source_id, is_tier0managed_by_system, skip_tep_routability_check) FROM stdin;

<#####> ACTIVATING tvmnsxlevcf [{"vmManagementIpAddress":"VM IP","vmHostname":"VM Name","sourceId":"####","id":"######"},{"vmManagementIpAddress":"VM IP","vmHostname":"fqdn of NSXT","sourceId":"#####","id":"#######"}] #####"

As we read in the above logs, the database entry for NSXT_Edge_Cluster status shows as ACTIVATING.

Follow the steps to correct the status to 'Active':

• Login to SDDC manager SSH session as VCF user
• Switch to root user using 'su' command.
  • Connect to database  psql -h localhost -U postgres
  • Navigate to platform: \c platform
  • List out status of NSXT_Edge and its relevant ID: platform=# SELECT status,id FROM nsxt_edge_cluster;
  • Change the status: platform=# UPDATE nsxt_edge_cluster SET status ='ACTIVE' WHERE id='obtained from above';


• Retry password management task from SDDC manager web UI.