How to cancel all queued BOSH tasks using director_ctl in Operations Manager

Products

• Operations Manager

Issue/Introduction

This articles pertains to Operations Manager v2.x and shows how to use director_ctl to cancel all queued BOSH tasks.

When there are tasks timing out or failing and too many BOSH tasks are queued for the Director, it is very difficult to cancel individual tasks using the following command:

bosh cancel task <#>

Environment

Product Version: 2.10

Resolution

WARNING: Make sure you are absolutely certain that the queued tasks are not affecting any ongoing deployments. If you have a currently running deployment DO NOT CONTINUE. Contact Tanzu Support for assistance.

The steps outlines in this article cancel all queued tasks and if you have a running deployment, it may leave it in an inconsistent and potentially broken state. Again, do not continue if you have a deployment that is in progress.

If you are using a recent version of Ops Manager 2.7+,  use the following command:

bosh cancel-tasks -s=queued 

If the previous command is not available, please follow the following steps:

1. Login to BOSH Director VM using vcap credentials (vcap credentials can be found on the Operations Manager Director tile, Credentials tab):

ssh vcap@<Director_IP> 

2. Switch to Superuser:

sudo su -

3. Enter the director console. The method of accessing the director console depends on the version of Operations Manager.

Operations Manager 2.1 or earlier

/var/vcap/jobs/director/bin/director_ctl console

Operations Manager 2.2 or later

/var/vcap/jobs/director/bin/console

4. Count the number of queued tasks with this command:

irb(main):001:0> Bosh::Director::Models::Task.where(state: "queued").count

5. Cancel all the queued BOSH tasks with this command:

irb(main):002:0> Bosh::Director::Models::Task.where(state: "queued").update(state: "cancelling")

For more information on BOSH tasks, refer to Reviewing Tasks.

Additional Information

Please refer to the BOSH documentation for more details on BOSH tasks.