greenplum command center always show segment down

Products

• VMware Tanzu Data

Issue/Introduction

In greenplum database v7.x, when the cluster does use mirror instance, sometimes gpcc will show certain segment is down.

Environment

gpdb 7.0 - 7.4.x

Cause

In gpdb 7.x, for a primary only cluster, when there is one segment is down, FTS service will add one entry in gp_configuration_history table, to show the history of segment down. Since it's a primary only cluster, after database restart, all failed segment can be started directly. After the database restart, FTS will not add an entry in gp_configuration_history table to mark the history of segment is up. From gpcc, it will use the latest records from gp_configuration_history table to show the status of each segment. Since there is no new entry for segment is up, gpcc will always show that segment as down.

Resolution

In gpdb v7.5.0, FTS will add a new entry for segment is up.  Besides that, a temp workaround is to manually add a new entry for segment is up or directly remove the old segment is down entry from gp_configuration_history table.