ABX action is stuck in a running status and cannot be canceled

Products

• VMware Aria Suite

Issue/Introduction

An ABX action is stuck in a running status and the option to cancel these action runs is greyed out.

Environment

Aria Automation 8.16.2

Cause

Could occur for various reasons, but one example is that the VMs the action run is supposed to run the script on no longer exist. 

Resolution

The Action Run can be manually canceled in the ABX database and if desired, it can them be deleted in the UI. 

Prerequisites:

You must back up all VMware Aria Automation appliances, at the same time - simultaneously for all nodes.
If you are taking the snapshots manually, you must start the snapshots of the second and the third node no more than 40 seconds after you start the snapshot for the first node.
When you back up the VMware Aria Automation appliance, disable in-memory snapshots and enable quiescing.

Steps to set the workflows as canceled:

1. Open the Action Run from each in Extensibility > Activity > Action Runs and get the ID that is listed at the top.
2. SSH into a vRA node and run
        vracli dev psql abx-db
3. At the database prompt, enter \x to turn on the extended view. 
4. Then find the Action runs by ID the validate their states (replace the id with the id of the action run you collected from the UI in the first step):
        SELECT id, name, run_state_value FROM abx_action_run WHERE id = '8a74808e9123e15801912441dabd0012';

It should return similar to this with a status of RUNNING:
   -[ RECORD 1 ]---+---------------------------------
id              | 8a74808e9123e15801912441dabd0012
name            | TEST
run_state_value | RUNNING
5. Then update the run_state_value to FAILED:
   UPDATE abx_action_run SET run_state_value = 'FAILED' WHERE id = '8a74808e9123e15801912441dabd0012';
6. Run the SELECT statement again to verify that tit was updated correctly:
7. SELECT id, name, run_state_value FROM abx_action_run WHERE id = '8a74808e9123e15801912441dabd0012';
8. Type \q and hit enter to quit postgres.
9. Check the UI and it will now be marked as failed.
10. If desired, after selecting the Action run, you can now Delete in the UI. 
       (You may want to delete the action run if it has been running for a while, and possibly logging a large amount of data, as it will take up space int he database.)