SDDC Manager UI unable to load workload domains & hosts information, error in operationsmanager.log/domainmanager.log - "Could not acquire change log lock. Currently locked by", error in vcf-commonsvcs.log - "502 Bad Gateway".
search cancel

SDDC Manager UI unable to load workload domains & hosts information, error in operationsmanager.log/domainmanager.log - "Could not acquire change log lock. Currently locked by", error in vcf-commonsvcs.log - "502 Bad Gateway".

book

Article ID: 374578

calendar_today

Updated On:

Products

VMware SDDC Manager

Issue/Introduction

- SDDC Manager UI unable to load workload domains and hosts information and more.

  • UI Stuck in INITIALIZING
    • unable to get ANY password via lookup passwords [no password expired, or changed] 
    • vcf-comonosvc.log logs under ERROR
      Invocation of init method failed; nested exception is liquibase.exception.CommandExecutionException: liquibase.exception.LockException: Could not acquire change log lock.  Currently locked by sddc_manager_fqdn (IP of SDDC manager)
  • Errors on SDDC Manager Dashboard page:
    • 0 Solutions.
    • 0 Workload Domains.

 

  • Errors on Inventory > Workload Domains page:
    • Failed to load all domains information. Could not reach the server, possible reasons could be network connectivity issue or server going down.
    • Failed to load cluster details. Could not reach the server, possible reasons could be network connectivity issue or server going down.

 

  • Errors on Inventory > Hosts page:
    • Failed to load host details. Could not reach the server, possible reasons could be network connectivity issue or server going down.
    • Failed to load host details. Unable to retrieve aggregated Network Pools info.
    • 502 Bad Gateway.

 

  • operationsmanager.log and domainmanager.log - "Could not acquire change log lock.  Currently locked by...":

XXXX-XX-XXTXX:XX:XX.XXX+0000 WARN  [vcf_om,0000000000000000,0000] [o.s.b.w.s.c.AnnotationConfigServletWebServerApplicationContext,main] Exception encountered during context initialization - cancelling refresh attempt: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'liquibase' defined in class path resource [com/vmware/vcf/operationsmanager/JpaConfig.class]: Invocation of init method failed; nested exception is liquibase.exception.CommandExecutionException: liquibase.exception.LockException: Could not acquire change log lock.  Currently locked by sddc_manager_fqdn (A.B.C.D) since X/X/XX, X:XX PM

  • vcf-commonsvcs.log - "502 Bad Gateway":

XXXX-XX-XXTXX:XX:XX.XXX+0000 ERROR [common,0000000000000000,0000] [c.v.e.s.t.services.TaskCollector,pool-1-thread-6] collect() failed
org.springframework.web.client.HttpServerErrorException$BadGateway: 502 Bad Gateway: "<html><EOL><EOL><head><title>502 Bad Gateway</title></head><EOL><EOL><body><EOL><EOL><center><h1>502 Bad Gateway</h1></center><EOL><EOL><hr><center>nginx</center><EOL><EOL></body><EOL><EOL></html><EOL><EOL>"

Environment

VMware Cloud Foundation 5.1.X
VMware Cloud Foundation 5.2.X

Cause

This issue is caused by a stale lock in one or more databases.

Resolution

Please open a case with VMware by Broadcom and reference the KB 374578

Powered by Wolken Software