Error: could not accept SSL connection: tlsv1 alert internal error

Products

• VMware Cloud Director

Issue/Introduction

• The VMware Cloud Director cell(s) are in-accessible via UI
• repmgr shows the cell(s) as failed when using:

sudo -i -u postgres repmgr cluster show

• The postgres service is in a stopped state when running:

systemctl status vpostgres.service

• Unable to start the postgres service
• Postgres logs located in /var/vmware/vpostgres/current/pgdata/log show the following errors: 

LOG: could not accept SSL connection: tlsv1 alert internal error
FATAL: the database system is shutting down
LOG: database system is shut down

Environment

10.x, 10.3.3

Cause

This issue can occur if the cells(s) lose network connectivity causing the postgres service to fail. 

Resolution

1. SHUTDOWN ALL CELLS, TAKE BACKUPS AND SNAPSHOTS PRIOR TO MAKING ANY DATABASE CHANGES. See Backup and Restore of VMware Cloud Director Appliance
   IF YOU DO NOT FEEL COMFORTABLE PERFORMING THESE STEPS CONTACT BROADCOM SUPPORT FOR ASSISTANCE.
2. Remove the Standby from the Repmgr Cluster. 

1. 1. Obtain the ID of the failed cell.

      # sudo -i -u postgres repmgr cluster show

   2. Un-register the failed cell from the Repmgr Cluster. 

      # sudo -i -u postgres repmgr standby unregister -f  /opt/vmware/vpostgres/current/etc/repmgr.conf --node-id=xxxxx

2. Delete the Postgres Database from the Failed Standby Cell. 

   1. Stop the vCD Services.

      # service vmware-vcd stop

   2. Stop the Postgres Service.

      # systemctl stop vpostgres.service

   3. Delete the Postgres Database from the failed standby cell.  

      # rm -rf /var/vmware/vpostgres/current/pgdata

3. Copy the Primary Cell Database to the Standby Cell. 

   # sudo -i -u postgres repmgr -h <primary_database_ip> -U repmgr -d repmgr -f /opt/vmware/vpostgres/current/etc/repmgr.conf standby clone

4. Start Postgres Service. 

   # systemctl start vpostgres.service

5. Register the Standby Cell with the Repmgr Cluster. 

   # sudo -i -u postgres repmgr -h <primary_database_ip> -U repmgr -d repmgr -f /opt/vmware/vpostgres/current/etc/repmgr.conf standby register --force

6. Verify the Standby Cell is registered to the Repmgr Cluster Successfully and you no longer see the error message or replication lag with the Standby Cell. 

    # sudo -i -u postgres repmgr cluster show

7. Start the vCD Services. 

   # service vmware-vcd start