Primary Cloud Director cell enter failed status after reboot

Products

• VMware Cloud Director

Issue/Introduction

• vcd and vpostgres services fail to start on the primary cell 
• Tenants are unable to access Cloud Director Portal
• /opt/vmware/vcloud-director/logs/cell-runtime.logs shows a postgres exception: 
  "org.postgresql.util.PSQLException: Connection to xx.xxx.x.xx:5432 refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections."
  
  
• postgres logs at /var/vmware/vpostgres/current/pgdata/log/ reports error :
  FATAL:  could not load pg_hba.conf
LOG:  database system is shut down

Environment

VMware Cloud Director

Cause

The issue was caused by IP addresses listed under the "IPv4 local connections" section of  pg_hba.conf file that were not setup with a proper subnet, causing Postgres to shutdown and stop working.

Resolution

To help solving the issue, we need to make sure that the ip addresses placed in the pg_hba.conf file contain an IP address with a valid subnet.

1. For example, checking the file /var/vmware/vpostgres/current/pgdata/pg_hba.conf you will see entries where the /32 subnet is missing:
   # IPv4 local connections:
   host    all             all             127.0.0.1/32            md5
   host    all             all             <IP_ADDRESS_1>/32       md5
   host    all             all             <IP_ADDRESS_2>          md5  < missing subnet
   host    all             all             <IP_ADDRESS_3>          md5  < missing subnet
   
   
2. Edit the pg_hba.conf file and add the /32 subnet:
   
   # IPv4 local connections:
   host    all             all             127.0.0.1/32           md5
   host    all             all             <IP_ADDRESS_1>/32      md5
   host    all             all             <IP_ADDRESS_2>/32      md5
   host    all             all             <IP_ADDRESS_3>/32      md5
   
   
   
3. Create a text file in /opt/vmware/appliance/etc/pg_hba.d/ containing the correct IPv4 local connection.
4. Reboot the cell to apply the changes.

Additional Information

Please refer this document for more information -   Configure External Access to the VMware Cloud Director Database