VCHA PostgreSQL replication is not in progress

Products

• VMware vCenter Server

Issue/Introduction

When enabling VCHA, the error message appears:

• "PostgreSQL replication is not in progress. For an initial deployment, PostgreSQL standby creation may take time depending on the database size. If this issue persists, verify that PostgreSQL is running on the passive node and that the passive node is reachable on the vCenter HA network."

 

• All VCHA nodes successfully deployed (Active/Passive/Witness).
• There are no issues in communication between the VCHA nodes - all are pingable, ports are open.
• PostgresSQL is running.
• On the Passive node, the /storage/seat partition is continuously growing in size.
• On the Passive node, in /var/log/vmware/vcha/repl_passive_setup.log, a re-occurring attempt to create base backup is failing with "Permission denied" on a file:

  • YYYY-MM-DDTHH:MM:SS.ZZZ INFO repl_passive_setup rc = [1], stdout = [], stderr = [pg_basebackup: initiating base backup, waiting for checkpoint to complete pg_basebackup: checkpoint completed pg_basebackup: write-ahead log start point: XX/XX000XXX on timeline 1 pg_basebackup: starting background WAL receiver pg_basebackup: created temporary replication slot "pg_basebackup_XXXXXXX"       33/24804963 kB (0%), 0/8 tablespaces (...sp/XX_ZZ_XXXXXXX/CCCC/DDDDD)    58783/24804963 kB (0%), 0/8 tablespaces (...sp/XX_ZZ_XXXXXXX/CCCC/DDDDD)   152445/24804963 kB (0%), 0/8 tablespaces (...sp/XX_ZZ_XXXXXXX/CCCC/DDDDD) .. .... 23791322/24804963 kB (95%), 6/8 tablespaces (...sp/XX_ZZ_XXXXXXX/CCCC/DDDDD) 23791323/24804963 kB (95%), 7/8 tablespaces (/storage/db/vpostgres/backup_label ) 23791323/24804963 kB (100%), 8/8 tablespaces pg_basebackup: error: could not get write-ahead log end position from server: ERROR:  could not open file "./postgresql.orig": Permission denied pg_basebackup: removing contents of data directory "/storage/db/vpostgres" pg_basebackup: removing contents of WAL directory "/storage/dblog/vpostgres/pg_xlog" pg_basebackup: changes to tablespace directories will not be undone

Environment

vCenter 8

 

Cause

The file postgresql.orig is likely a manually created file to which PostgreSQL does not have permissions.

Resolution

• On a VCHA Active Node:

  • # cd /storage/db/vpostgres

  • # ls -la

     

• Check output for files that are owned by root:root rather than vpostgres:vpgmongrp. In the log snippet above we can see the filename "postgresql.orig". Usually, these are files that were created manually as a backup when making changes, so may be called for example, postgresql.orig or postgresql.backup, or something else.

• Move, or delete the file.
• Restart VCHA Passive node.