Concourse for VMware Tanzu job gets stuck in pending on max-in-flight check

Products

• Concourse for VMware Tanzu

Issue/Introduction

Your Concourse for VMware Tanzu job gets stuck in pending on max-in-flight check. The builds table in the DB is showing job as aborted. The column "serial=true" in the builds table for the job or build is hanging and "scheduled=true". The scheduler is looking at that pending build and since serial is set to true, it gets stuck on the check if max-in-flight is reached.

Those are the symptoms of the bug described here: https://github.com/concourse/concourse/issues/4065

The fix will be provided in newer version Concourse for VMware Tanzu 6.0.x.

Environment

Product Version: 5.2

Resolution

The workaround is to run the steps below:

1. Get pipeline id from pipeline table, use the following query: 
 

Select id, name from pipelines;

 

Note: Match the name of the pipeline with the job that is hanging.

2. Get correct id from builds table using the pipeline-id and build number:
 

select id from builds where pipeline_id=3 AND name='7';

 

Note: Pipeline-id is retrieved from the first step and name is the build number that is hanging.

3. Update the scheduled value to false for the correct ID of the build:

UPDATE builds SET scheduled = false WHERE id = 2; 

This will update the scheduled field on that pending build to be false and free up the max in flight check to make it able to schedule that build again.