Queries fail with “Failed to setup ic_proxy interconnect” when gp_interconnect_proxy_addresses is incomplete

Products

• VMware Tanzu Greenplum
• VMware Tanzu Greenplum / Gemfire

Issue/Introduction

After upgrading Greenplum Database to 6.31 or later, the database may throw interconnect errors when gp_interconnect_type is set to proxy.

Customers may observe the following symptoms:

• All queries fail, including simple SELECT statements.

• Internal GUC retrieval fails (for example, gpconfig -s).

• GPCC may report interconnect-related errors.

• The database appears to start successfully but no queries can be executed.

The following error is consistently reported:

ERROR: Failed to setup ic_proxy interconnect
DETAIL: expected <N>, found <N-1>
HINT: Please configure gp_interconnect_proxy_addresses guc to include the proxy details of all segments.

Cause

Starting with Greenplum 6.31, stricter validation was introduced for the proxy interconnect configuration.

If the gp_interconnect_proxy_addresses GUC does not include proxy details for all segments, Greenplum now raises an error early instead of allowing queries to proceed.

This situation commonly occurs when a standby master is removed and re-initialized as the standby master receives a new DBID. The gp_segment_configuration reflects the new DBID and gp_interconnect_proxy_addresses still contains a stale DBID and is missing a required DBID.

Although all segments may be running and the cluster may start successfully, the runtime validation fails, causing query execution to stop.

Note:

Earlier versions (for example, 6.27.x) were more permissive and could continue to operate with incomplete or stale proxy configuration. This behavior changed in 6.31 to resolve an issue where queries could hang when gp_interconnect_type=proxy was set without specifying proxy details for all segments in the gp_interconnect_proxy_addresses GUC. A validation check has now been added to raise an error early if the proxy configuration is incomplete.

Resolution

Workaround:

As a temporary workaround, change the interconnect type to udpifc:

gpconfig -c gp_interconnect_type -v udpifc
gpstop -u

 

This allows queries to run but does not resolve the underlying proxy configuration issue.

 

Resolution:

Ensure that gp_interconnect_proxy_addresses includes proxy entries for all DBIDs present in gp_segment_configuration.

Steps:

1. Identify the current DBIDs:

SELECT * FROM gp_segment_configuration ORDER BY dbid;

 

2. Update gp_interconnect_proxy_addresses to:

• Remove any stale DBIDs

• Add any missing DBIDs

3. Make new configuration settings active:

gpstop -u