"FATAL: no pg_hba.conf entry for replication connection." error during Tanzu Greenplum v6.19.0 installation

Products

• VMware Tanzu Greenplum

Issue/Introduction

While running gpinitsystem in installing new cluster on Tanzu Greenplum v6.19.0 and Greenplum v6.19.1, the installation fails with the following error:

20220126:09:18:30:000500 gpcreateseg.sh:mdw1:gpadmin-[INFO]:-Commencing remote /bin/ssh sdw1 export GPHOME=/usr/local/greenplum-db-6.19.0; . /usr/local/greenplum-db-6.19.0/greenplum_path.sh; rm -rf /data2/mirror/gpseg3; /usr/local/greenplum-db-6.19.0/bin/pg_basebackup --xlog-method=stream --slot='internal_wal_replication_slot' -R -c fast -E ./db_dumps -E ./gpperfmon/data -E ./gpperfmon/logs -D /data2/mirror/gpseg3 -h sdw2 -p 1026 --target-gp-dbid 9;
pg_basebackup: could not connect to server: FATAL:  no pg_hba.conf entry for replication connection from host "192.168.10.8", user "gpadmin", SSL off

The issue is caused during the creation of mirror segments. It is unable to connect to the already created primary segments. This is because the pg_hba.conf file for the primary segments is created incorrectly. 

The MIRROR_ADDRESSES/PRIMARY_ADDRESSES are picked up in one line rather than 2. As a result, the required IP needed to access the segment psql is not added to the pg_hba.conf.

This can happen if you have multiple network interfaces and the first interface is not reachable by other segment hosts.

For example:

eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 10.0.2.15  netmask 255.255.255.0  broadcast 10.0.2.255

eth1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.99.123  netmask 255.255.255.0  broadcast 192.168.99.255

This results in pg_hba.conf:

host replication gpadmin samehost trust
host replication gpadmin 10.0.2.15/32 trust
host replication gpadmin 10.0.2.15/32 trust

Whereas, the results in pg_hba.conf should be:

host replication gpadmin samehost trust
host replication gpadmin 10.0.2.15/32 trust
host replication gpadmin 192.168.99.123/32 trust
host replication gpadmin 10.0.2.15/32 trust
host replication gpadmin 192.168.99.122/32 trust

In a setup or configuration where the first interface is reachable by other segments, you do not experience this problem.

Environment

Product Version: 6.18

Resolution

This is a bug in the Tanzu Greenplum v6.19.0 and v6.19.1. Check a for a new version of v6.19 release notes to see if bug 32042 is fixed in that version. 

To work around this issue, install the cluster at v6.18.2 which does not have this issue.