SDDC Manager password operations are not allowed because of a failed password task

Products

• VMware Cloud Foundation

Issue/Introduction

• When attempting to perform any password-related task in Password Manager, one or more of the following errors may be encountered:

  • “This operation is not allowed because Password Manager operation(s) are working on related resources.”

  • Banner message on the VCF UI:

    "The scheduled auto-rotate workflow could not run due to another operation blocking the resources."

  • “The operation is not allowed because the system lock is held by a Password Manager operation in progress.”

• Similar errors are seen in the operationsmanager.log
  
  /var/log/vmware/vcf/operationsmanager/operationsmanager.log
  yyyy-mm-ddThh:mm:ss DEBUG [vcf_om,40#######ef,f8fb] [c.v.e.s.e.h.LocalizableRuntimeExceptionHandler,http-nio-127.0.0.1-7300-exec-4] Handler Error Response: {"errorCode":"PASSWORD_MANAGER_UPDATE_IN_PROGRESS","arguments":["d68f####-####-####-###99"],"message":"Password update task d68f####-####-####-###99 is in progress/failed state, retry after sometime.","remediationMessage":"Wait for currently in-progress transaction to complete as succeeded/cancelled.","referenceToken":"GURKAF"}

Environment

• VCF 5.x
• VCF 9.x

Cause

• There might be a password rotation task failed on individual components due to other reasons(Password expired, SSH disabled etc)
• Reported failed workflow of password manager holds a lock due to task being in FAILED or IN_PROGRESS state 

Note: The task should either be in USER_CANCELLED or SUCCESSFUL state

Resolution

If there is a banner on the Password Manager page, Cancel the task from the UI.