SDDC Manager upgrade fails: Workflow changes to 'CANCELLED' shortly after being triggered

Products

• VCF Operations

Issue/Introduction

• Initiating an SDDC Manager upgrade via the UI fails almost immediately.
• Task status in the SDDC Manager UI shows CANCELLED.
• Task info displays: Upgrade SDDC_MANAGER - [####] using BUNDLE - SDDC_MANAGER:9.1.x.
• Running psql -h localhost -U postgres -d platform -c "select * from lock;" shows existing entries in the lock table, such as resources of type DOMAIN, SYSTEM, NSX, or VCENTER held by OPERATIONS_MANAGER

Environment

VMware Cloud Foundation 9.x

SDDC Manager

Cause

The issue is caused by stale resource locks remaining in the SDDC Manager platform database from a previous or failed workflow. These locks prevent the Lifecycle Management (LCM) service from acquiring the necessary deployment-level lock to proceed with the upgrade.

Resolution

To resolve this and allow the upgrade to proceed, all four locks should be cleared. Follow these steps:

Important: Ensure you have a fresh snapshot of the SDDC Manager VM before performing the steps below.

1. Log in to the SDDC Manager appliance via SSH as the vcf user and switch to root.
2. Run the following command to clear all stale locks from the database: psql -h localhost -U postgres -d platform -c "truncate lock;"
3. Restart the Lifecycle Management service: systemctl restart lcm
4. Log back into the SDDC Manager UI and re-attempt the upgrade.