Identity Manager (vIDM) admin UI is unable to load the Roles tab and ACS Health shows red

Products

• VMware Aria Suite

Issue/Introduction

When attempting to access the 'Roles' tab within the admin UI the following error is shown:

"An unexpected error has occurred. Please try again later."

The ACS Health section of System Diagnostics page shows RED with the following error:

"Error when connecting to the application"

 

The accesscontrol-service.log file under /opt/vmware/horizon/workspace/logs shows the following error:

2024-08-21 20:25:17,972 GMT ERROR accesscontrol (localhost-startStop-8) [;;;] org.springframework.boot.SpringApplication - Application run failed org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'liquibase' defined in class path resource [com/vmware/vidm/accesscontrol/db/DbDataStoreAutoConfiguration.class]: Invocation of init method failed; nested exception is liquibase.exception.LockException: Could not acquire change log lock.  Currently locked by XX.XX.XX.XX (XX.XX.XX.XX) since 8/19/24 5:50 AM

2024-08-21 20:25:17,982 GMT INFO accesscontrol (localhost-startStop-8) [;;;] com.vmware.vidm.common.http.context.listener.AbstractBootstrapServletContextListener - Shutting down micro service

Environment

Identity Manager 3.3.x

Cause

There is a lock on the Access Control Service database table preventing the service from starting properly

Resolution

To resolve this issue we need to remove the ACS database change lock

NOTE - Please ensure valid backups / snapshots are taken prior to performing the steps below

1. Login to all vIDM nodes via SSH as root user
2. Shutdown horizon-workspace service on all nodes

service horizon-workspace stop

3. On the master node connect to Postgres by running the following command

psql -U postgres saas

It will request Password for user postgres:

Fetch the password using below command 
cat /usr/local/horizon/conf/db.pwd

4. Run the following command to confirm the lock

select * from saas.ACS_DATABASECHANGELOGLOCK;

5. If there is a lock present, release the lock with this command

update saas.ACS_DATABASECHANGELOGLOCK set locked='f', lockgranted=NULL, lockedby=NULL where id=1;

6. Start the horizon-workspace service on master node. Wait 30 seconds and start horizon-workspace service on other two nodes

service horizon-workspace start

7. Access the admin UI once again. The Roles tab should now be accessible and the ACS health should now also show GREEN