Failover to Standby
Article ID: 296354
Updated On:
Products
Issue/Introduction
How to Failover to Standby Node
Environment
Product Version: 6.7
Resolution
Note: The following operation is very risky. It is strongly recommended to contact Broadcom Support for assistance.
Activate Standby Master as Active Master
1. Make sure standby host is configured and is in sync with the current master (Summary state: Synchronized)
$ gpstate -f >>>> if not synchronized then run “gpinitstandby -n” to resynchronize it.
2. Stop the current Master host by using the following:
$ gpstop -m
3. Login to the Standby Master host which you are activating $ ssh smdw
4. Following parameter should be set on smdw. You can verify using the “env” command.
b) PGPORT (default value 5432)
Can use below example if defaults are set:
$ export MASTER_DATA_DIRECTORY=/data/master/gpseg-1
$ export PGPORT= 54325
Activate standby by using the below command:
gpactivatestandby -d $MASTER_DATA_DIRECTORY
Where -d specifies the data directory of the master host you are activating. The above command will prompt for approval and will then activate this standby master as current master.
5. After the utility finishes, run gpstate to check the status and run a few DDL/DML to verify on smdw:
$ gpstate -e
Add original master as Standby Master
1. Ensure the original master host (in our case its mdw) is in dependable running
2. On the original master host (mdw), move/rename the data directory, gpseg-1 (add timestamp to the name for reference is needed later).
[gpadmin@multi-m master]$ pwd /data/master [gpadmin@multi-m master]$ ls gp_6.12.0_20201111223347-1 [gpadmin@multi-m master]$ mv gp_6.12.0_20201111223347-1/ gp_6.12__bkp_2020_11_11 [gpadmin@multi-m master]$ ls gp_6.12__bkp_2020_11_11
You can remove the backup directory once the standby is successfully configured and verified.
3. Initialize a standby master on the original master host.
For example, run this command from the current master host, smdw:
$ gpinitstandby -s mdw
4. After the initialization completes, check the status of standby master.
You have to run gpstate -f from current master i.e. smdw to check the status:
gpstate -f 20201111:22:45:14:008860 gpstate:multi-s:gpadmin-[INFO]:-Starting gpstate with args: -f 20201111:22:45:14:008860 gpstate:multi-s:gpadmin-[INFO]:-local Greenplum Version: 'postgres (Greenplum Database) 6.12.0 build commit:4c176763c7619fb678ce38095e6b3e8fb9548186' 20201111:22:45:14:008860 gpstate:multi-s:gpadmin-[INFO]:-master Greenplum Version: 'PostgreSQL 9.4.24 (Greenplum Database 6.12.0 build commit:4c176763c7619fb678ce38095e6b3e8fb9548186) on x86_64-unknown-linux-gnu, compiled by gcc (GCC) 6.4.0, 64-bit compiled on Oct 28 2020 19:42:15' 20201111:22:45:14:008860 gpstate:multi-s:gpadmin-[INFO]:-Obtaining Segment details from master... 20201111:22:45:14:008860 gpstate:multi-s:gpadmin-[INFO]:-Standby master details 20201111:22:45:14:008860 gpstate:multi-s:gpadmin-[INFO]:----------------------- 20201111:22:45:14:008860 gpstate:multi-s:gpadmin-[INFO]:- Standby address = multi-m 20201111:22:45:14:008860 gpstate:multi-s:gpadmin-[INFO]:- Standby data directory = /data/master/gp_6.12.0_20201111223347-1 20201111:22:45:14:008860 gpstate:multi-s:gpadmin-[INFO]:- Standby port = 3002 20201111:22:45:14:008860 gpstate:multi-s:gpadmin-[INFO]:- Standby PID = 13123 20201111:22:45:14:008860 gpstate:multi-s:gpadmin-[INFO]:- Standby status = Standby host passive 20201111:22:45:14:008860 gpstate:multi-s:gpadmin-[INFO]:-------------------------------------------------------------- 20201111:22:45:14:008860 gpstate:multi-s:gpadmin-[INFO]:--pg_stat_replication 20201111:22:45:14:008860 gpstate:multi-s:gpadmin-[INFO]:-------------------------------------------------------------- 20201111:22:45:14:008860 gpstate:multi-s:gpadmin-[INFO]:--WAL Sender State: streaming 20201111:22:45:14:008860 gpstate:multi-s:gpadmin-[INFO]:--Sync state: sync 20201111:22:45:14:008860 gpstate:multi-s:gpadmin-[INFO]:--Sent Location: 0/14000000 20201111:22:45:14:008860 gpstate:multi-s:gpadmin-[INFO]:--Flush Location: 0/14000000 20201111:22:45:14:008860 gpstate:multi-s:gpadmin-[INFO]:--Replay Location: 0/14000000 20201111:22:45:14:008860 gpstate:multi-s:gpadmin-[INFO]:--------------------------------------------------------------
The below steps are similar to section A, but in this case the hostname of master and standby are interchanged:
6. Stop Greenplum Database master instance on the standby (smdw) master.
$ gpstop -m
7. Activate standby by using below command after logging in to mdw:
ssh mdw $ gpactivatestandby -d $MASTER_DATA_DIRECTORY
After the utility finishes, run gpstate to check the status and run few DDL/DML to verify on mdw:
$ gpstate -e
8. On the standby master host, move/rename the data directory (add timestamp to the name for reference is needed later).
You can remove the backup directory once the standby is successfully configured and verified
Add standby again to revert into the initial state and check with gpstate -f:
$ gpinitstandby -s smdw $ gpstate -f
Feedback
