gpexpand fails during "Creating segment template" stage with no "pg_hba.conf entry for replication connection from host" error

Products

• VMware Tanzu Greenplum

Issue/Introduction

When attempting gpexpand, the error below occurs during the "Creating segment template" stage:

20230706:16:52:05:013002 gpexpand:gp00:gpadmin-[INFO]:-local Greenplum Version: 'postgres (Greenplum Database) 6.23.1 build commit:2731a45ecb364317207c560730cf9e2cbf17d7e4'
20230706:16:52:05:013002 gpexpand:gp00:gpadmin-[INFO]:-master Greenplum Version: 'PostgreSQL 9.4.26 (Greenplum Database 6.23.1 build commit:2731a45ecb364317207c560730cf9e2cbf17d7e4) on x86_64-unknown-linux-gnu, compiled by gcc (GCC) 6.4.0, 64-bit compiled on Feb 7 2023 22:21:34'
20230706:16:52:05:013002 gpexpand:gp00:gpadmin-[INFO]:-Querying gpexpand schema for current expansion state
20230706:16:52:08:013002 gpexpand:gp00:gpadmin-[INFO]:-Heap checksum setting consistent across cluster
20230706:16:52:09:013002 gpexpand:gp00:gpadmin-[INFO]:-Syncing Greenplum Database extensions
20230706:16:52:10:013002 gpexpand:gp00:gpadmin-[INFO]:-The following packages will be installed on gp03: pivotal-greenplum-backup-restore-1.27.0-gp6-rhel-x86_64.gppkg
20230706:16:52:14:013002 gpexpand:gp00:gpadmin-[INFO]:-Locking catalog
20230706:16:52:14:013002 gpexpand:gp00:gpadmin-[INFO]:-Locked catalog
20230706:16:52:14:013002 gpexpand:gp00:gpadmin-[INFO]:-Creating segment template
20230706:16:52:14:013002 gpexpand:gp00:gpadmin-[ERROR]:-gpexpand failed: ExecutionError: 'non-zero rc: 1' occurred. Details: 'pg_basebackup -c fast -D /data/master/gpexpand_07062023_13002 -h greenplum_master_0 -p 5332 --xlog --target-gp-dbid 69644 -E ./db_dumps -E ./gpperfmon/data -E ./gpperfmon/logs -E ./promote --progress --verbose' cmd had rc=1 completed=True halted=False
stdout=''
stderr='pg_basebackup: could not connect to server: FATAL: no pg_hba.conf entry for replication connection from host "X.X.X.X", user "gpadmin", SSL on
FATAL: no pg_hba.conf entry for replication connection from host "X.X.X.X", user "gpadmin", SSL off

Environment

Product Version: 6.23

Resolution

This error occurs when the master/coordinator pg_hba.conf file is missing the replication entries for itself, which is a requirement for gpexpand to be successful. These lines are required for expansion even if mirroring is not enabled on the system.

See below for example master/coordinator pg_hba.conf lines, with 10.0.34.111 being the master/coordinator IP:

host     replication gpadmin         samehost       trust
host     replication gpadmin         10.0.34.111/32       trust