Postgres does not come up due to missing delegate IP on vIDM cluster.

Products

• VCF Operations

Issue/Introduction

• VMware Identity Manager cluster failure. All three nodes are down, and the 'vpostgres' service continuously stops itself.
  
  

  /etc/init.d/vpostgres status

   
• The 'pgService' is reported down on all nodes, and the 'poolnodes' command fails to execute successfully.
  
  

  /etc/init.d/pgService status
  poolnodes

   
• OpenSearch is not running on any node, and attempts to start it time out on secondary nodes.
  
  

  /etc/init.d/opensearch status

   
• The delegate IP address is not assigned to the primary node. Running the command below on the 0primary shows not 'eth0:0' interface configured.
  

  ifconfig

Environment

• VMware Identity Manager 3.3.7

Cause

The cluster failure is caused by the absence of the delegate IP address on the primary node preventing cluster communication, which subsequently results in Liquibase database locks that prevent the OpenSearch service from starting.

Resolution

• Reassign the delegate IP to the primary node, as listed in VMware Aria Suite Lifecycle, using the following command:

  ifconfig eth0:0 inet delegateIP netmask <NetMask_IP>

• Restart the horizon-workspace service:

  service horizon-workspace restart

• Force release the Liquibase database locks using the hznAdminTool:

  /usr/sbin/hznAdminTool liquibaseOperations -forceReleaseLocks

• Start the OpenSearch service:

  /etc/init.d/opensearch start

• Monitor the cluster health API until the status transitions from red to yellow to green:

   http://localhost:9200/_cluster/health?pretty=true

Additional Information

• Delegate IP assignment
• Liquibase locks