Request stuck in "In Progress" and cannot be cancelled in Aria Automation 8.x

Products

• VCF Operations/Automation (formerly VMware Aria Suite)

Issue/Introduction

Day 2 actions (e.g., Reboot, Reset, Power Off) initiated on a deployment remain in an "In Progress" state indefinitely.

The option to "Cancel" the request is unavailable, greyed out, or fails to stop the process.

New actions cannot be performed on the deployment because the previous action is blocking execution.

Environment

VMware Aria Automation 8.x (Automation Assembler / Service Broker)

Applies to Single Node and Clustered environments.

Resolution

Warning: This procedure involves modifying the Aria Automation database directly.

You must back up all VMware Aria Automation or Orchestrator appliances, at the same time - simultaneously for all nodes. 
If you are making the snapshots manually, you must start the snapshots of the second and the third node not more than 40 seconds after you start the snapshots for the first node.
If the quiesced state was not achieved for all 3 nodes within the ~40 seconds time frame, the following strings will be found in the logs: "Freeze synchronization failed" and "Sync failed, making inconsistent snapshot".
Run the following command from one of the nodes to filter for all vmtoolsd messages: journal ctl --identifier=vmtoolsd
When you back up the VMware Aria Automation or Orchestrator appliance, disable in-memory snapshots and enable quiescing.

This procedure requires root access to the appliance.

 

Procedure

1. SSH into the primary Aria Automation appliance node as root.
2. Connect to the database interface:

   vracli dev psql

3. Switch to the Catalog database context:

   \c catalog-db

4. Identify the Deployment ID for the stuck resource (retrievable from the URL in the UI when viewing the deployment details).
5. Verification Step: Run the following SELECT query to confirm the stuck request exists for that specific deployment. Replace the UUID placeholders with the actual Deployment ID:

   SELECT id, name, status, created_at, resource_name 
   FROM dep_request 
   WHERE status = 'INPROGRESS' 
   AND deployment_id = '########-####-####-####-############';

6. Correction Step: Once confirmed (ensure only the expected rows are returned), run the UPDATE command to mark the request as FAILED:

   UPDATE dep_request 
   SET status = 'FAILED' 
   WHERE status = 'INPROGRESS' 
   AND deployment_id = '########-####-####-####-############';

7. Exit the database session:

   \q

Validation

1. Log in to the Aria Automation UI (Service Broker).
2. Navigate to the Requests tab or the specific Deployment history.
3. Refresh the page.
4. Verify that the previously stuck action now appears as "Failed".
5. Verify that new Day 2 actions can now be initiated on the deployment.