VMware Identity Manager fails to start with error: Could not acquire change log lock
search cancel

VMware Identity Manager fails to start with error: Could not acquire change log lock

book

Article ID: 370272

calendar_today

Updated On:

Products

VMware Aria Suite

Issue/Introduction

Accessing VIDM  UI in a browser returns a HTTP 404, or 502, or 503 “Bad Gateway” or red error banner. 

The Following error can be seen on Horizon.log,

 

ERROR (localhost-startStop-1) [;;] org.springframework.web.context.ContextLoader - Context initialization failedorg.springframework.beans.factory.BeanCreationException: Error creating bean with name 'liquibase' defined in class path resource [spring/datastore-wireup.xml]: Invocation of init method failed; nested exception is liquibase.exception.LockException: Could not acquire change log lock. Currently locked by <ipAddressHoldingLock> (<ipAddressHoldingLock>)

Authentication to applications through WS1 VIDM fails

 

Environment

VMware Identity Manager 3.3.x

Cause

When performing a database migration performing a database migration, or if the application experiences an ungraceful shutdown, or if there are network disruptions between nodes and VIDM, it can result in a stale database lock record in the saas.databasechangeloglock.

Resolution

The following steps can be taken to manually remove the lock on the VIDM Database:

  1. Take a valid Snapshot /backup of the VIDM cluster nodes. 

  2.  Log in using root credentials to a console session on each VIDM appliance node and stop the horizon-workspace service by running the following command:
    # service horizon-workspace stop

     

  3. Connect to the VIDM database on master node and run the following query to verify if the database is in a Locked state:
    export PGPASSWORD=`cat /usr/local/horizon/conf/db.pwd`
    /opt/vmware/vpostgres/current/bin/psql -U postgres saas
    
    saas# SELECT * FROM saas.DATABASECHANGELOGLOCK;
    
     If database locked will show t for true, the following steps can be taken to release the lock manually
     id | locked | lockgranted | lockedby
    ----+--------+-------------+----------
     1 | t            |   xxxxxxxxx  | xxxxxxx

     

  4. Execute the following update to release the lock on the primary node:
    UPDATE saas.DATABASECHANGELOGLOCK set locked=FALSE, lockgranted=NULL, lockedby=NULL where id=1;

    Re-run Step 3 to verify that the lock has been released (LOCKED should have the value f)


  5. Start the horizon-workspace service by running the following command on each node. It is highly recommended that you wait 2 minutes between running this service on one node and the next.
    # service horizon-workspace start