Found active workflow for Other, Description: Password management operation in progress

Products

• VMware Cloud Foundation

Issue/Introduction

Symptoms:

• Reviewing password management in the SDDC Manager UI, a password rotation workflow is failed on  "Password management operation failed in pre-validation stage".
• When trying to collect SOS logs, it reports "active workflow for OTHER, Description: Password management operation in progress. Please wait on completion."
• In /var/log/vmware/vcf/operationsmanager/operationsmanager.log, it reports "The credentials were incorrect or the account specified has been locked.","error_code":403"

Environment

VMware Cloud Foundation 4.x

VMware Cloud Foundation 5.x

VMware Cloud Foundation 9.0.0.0/9.0.1.0

Resolution

1. Open a SSH to the SDDC Manager VM and change to root:

su - 

 

2. Run the following command to get a new Token:

TOKEN=$(curl -d '{"username" : "<SSOAdminUsername>", "password" : "<SSOUserpassword>"}' -H "Content-Type: application/json" -X POST http://127.0.0.1/v1/tokens | jq -r '.accessToken')

 

3. Run the below command to fetch all the credentials tasks:

curl 'http://localhost/v1/credentials/tasks' -i -X GET -H 'Content-Type: application/json' -H "Authorization: Bearer $TOKEN"

 

4. From the output of the above curl command, retried the failed credential update/rotate operation.

 

5. Note the id of the failed update/rotate operation from above and execute the cancellation using the command below:

curl http://localhost/v1/credentials/tasks/<id-of-failed-task> -X DELETE -H "Authorization: Bearer $TOKEN" | jq

 

Note: Token created in step 1 remains valid for one hour.