Multiple tasks in SDDC Manager fails with Error: ‘This operation is not allowed because SDDC Manager Backup Operation(s) are working on related resources.’

Products

• VMware Cloud Foundation

Issue/Introduction

• When attempting to perform domain-related operations in SDDC Manager such as host commissioning or VI domain creation or attempting to Remediate and Update Password you may encounter the following error message:

  This operation is not allowed because SDDC Manager Backup Operation operation(s) are working on related resources.

  
• SDDC Manager Backup is failing with the error:

  SDDC Manager Deployment lock to perform Backup & Restore Operation

  
• In some instances the "Commission hosts" option also gets greyed out in the SDDC manager UI:
  

Environment

• Vmware Cloud Foundation 4.5
• VMware Cloud Foundation 5.0
• VMware Cloud Foundation 9.0

Cause

This issue occurs when a restore operation in SDDC Manager fails to release a deployment lock that was previously acquired by a backup operation.
As a result, any new deployment-related operations are blocked due to the lingering lock entry.

Resolution

To work around this issue, remove the deployment lock.

Important: Always take a snapshot of the SDDC Manager appliance before performing any database-level modifications.

• SSH into the SDDC Manager appliance as the vcf user and then switch to root using:

  # su -

• Verify that a lock exists by running this command:

  # psql -h localhost -U postgres -d platform -c "select * from lock;"

• Delete the DB lock entry with:

  # psql -h localhost -U postgres -d platform -c "delete from lock where resource_type='deployment';"

Following these steps, domain-related operations in SDDC Manager should proceed without issue.

Note: If the issue persists for Remediate or Update Password operations, it is likely due to failed password tasks in the database.
Refer to the following KB article for instructions on clearing failed password operations:  Error: Password Manager operations working on related resources - SDDC Manager for steps to solve this.