node status DEAD in pgpool command due to corrupted pgService file in VIDM

Products

• VCF Operations/Automation (formerly VMware Aria Suite)

Issue/Introduction

• Attempts to start services manually using the service pgService start result in no response or errors.
• Running show poolnodes or pgpool status commands shows a node as "DEAD".

su root -c "echo -e 'password'|/usr/local/bin/pcp_watchdog_info -p 9898 -h localhost -U pgpool"

<Host1>:9999 Linux <Host1> <Host1> 9999 9000 4 MASTER
<Host2>:9999 Linux <Host2> <Host2> 9999 9000 7 STANDBY
<Host3>:9999 Linux <Host3> <Host3> 9999 9000 7 DEAD

 

Environment

VMware Identity Manager 3.3.7

Cause

/etc/init.d/pgService configuration file on the affected node is corrupted or has a file size of 0 bytes.

Resolution

To resolve the issue, perform the following steps:

1. Take a snapshot of all vIDM cluster nodes.
2. Log in to a healthy node in the cluster via SSH as root.
3. Identify the size and integrity of the service file: ls -l /etc/init.d/pgService.
4. Log in to the affected node and check the same file. If it is 0 bytes, proceed.
5. Copy the /etc/init.d/pgService file from the healthy node to the affected node (e.g., using scp).
6. On the affected node, ensure the file has correct permissions: chmod +x /etc/init.d/pgService.
7. Start the service manually to test: service pgService start.
8. Perform a graceful cluster restart via Aria Suite Lifecycle to ensure all nodes synchronize correctly.