GPDR recovery process 'error occurred while restoring database cluster: segment -1 is down in unexpected control file state "in archive recovery"'

Products

• VMware Tanzu Greenplum

Issue/Introduction

The following error is encountered while the GPDR recovery process tries to recover DR site AFTER there has been a coordinator/standby failover and fail-back:

gpdr:gpadmin:<DRServerName>:3323595-[ERROR]:-error occurred while restoring database cluster: segment -1 is down in unexpected control file state "in archive recovery"; check GPDB logs for more information

 

Resolution

The standby coordinator host doesn't seem to be configured correctly for GPDR as it wasn't able to access the repository via pgbackrest APIs. This suggests that the /usr/local/gpdr/configs/pgbackrest-seg-1.conf was missing on the standby coordinator host.

RCA:

'gpdr configure backup' does not copy over the pgbackrest-seg-1.conf file over to the standby coordinator host.

The workaround:

Manually copy over the pgbackrest-seg-1.conf file from the coordinator host to the standby coordinator host.

To fix the recovery error the client can choose one of the following:

1. Run a full backup rather than an incremental and continue from the new full backup.

or

2. Log a ticket with support referencing this KB article, the steps for this option temporarily corrupts the GPDR repository and should only be performed by R&D or Support.