vRealize Automation ABX service fails with CrashLoopbackOff after deploy.sh --onlyClean command

Products

• VMware Aria Suite

Issue/Introduction

The purpose of this article is to document a known issue and the workaround.

Symptoms:
After running the deploy.sh --onlyClean command the abx-service-app fails in a CrashLoopBackoff state.

kubectl get pods -n prelude -w

abx-service-app CrashLoopbackOff
approval-service-app Init:0/2
catalog-service-app Init:0/2

The abx-service-app logs indicate the service is waiting on a lock:

2021-10-19T21:29:37.746Z INFO abx-service [host='abx-service-app-5d898654bf-nmzxg' thread='main' user='' org='' trace='' component='Slf4jLogger' method='info:42' actionRunId='' context='' tokenId='' parentTokenId=''] l.lockservice.StandardLockService - Waiting for changelog lock....
Waiting for changelog lock....

Environment

VMware vRealize Automation 8.4.x
VMware vRealize Automation 8.3.x
VMware vRealize Automation 8.2.x
VMware vRealize Automation 8.0.x
VMware vRealize Automation 8.5.x
VMware vRealize Automation 8.x
VMware vRealize Automation 8.1.x

Cause

This can be caused by a database lock on the public.flw_ev_databasechangeloglock table.

Resolution

This is a known issue affecting vRealize Automation.  Please subscribe to this article to be notified of updates.

To workaround this issue, please clear the database lock with the following steps:

1. Run `deploy.sh`.
2. After postgres starts (all three postgres pods are ready), run "vracli dev psql abx-db" and enter the following query:

UPDATE public.flw_ev_databasechangeloglock SET locked=false, lockgranted=null, lockedby=null;

3. Now `deploy.sh` should finish successfully and the abx-service-app pod should start properly.