Interconnect Error Writing an Outgoing Packet: Network is unreachable" seen during gpcheckcat Execution
Article ID: 295466
Updated On:
Products
Issue/Introduction
During a gpcheckcat execution - queries are failing with "ERROR: Interconnect error writing an outgoing packet: Network is unreachable".
[ERROR] executing: Missing or extraneous entries check for pg_type_encoding Execution error: ERROR: Interconnect error writing an outgoing packet: Network is unreachable (seg0 slice4 sdw1:46000 pid=21056) DETAIL: error during sendto() call (error:101). For Remote Connection: contentId=-1 at 10.110.121.160:32399
The segment (Seg0 sdw1:46000) is trying to connect to the Master (contentId=-1) using IP 10.110.121.160.
Environment
Cause
As you can notice from the Error and description above, the master host was unreachable when the segment was trying to connect to the IP: 10.110.121.160.
The reason for this error is that /etc/hosts on the master is pointing towards external IP address that cannot be accessed from the segment. Example of the bad entries are mentioned below from /etc/hosts:
10.110.121.160 mdw --> Only this IP is from ETH0, rest below are from ETH1 and in this case other segments did not have a valid route to it. 172.28.19.228 sdw1 172.28.19.230 sdw2 172.28.19.229 sdw3 172.28.19.227 smdw
Resolution
In this scenario, the entries in /etc/hosts on the master have an IP that is unreachable.
Ensure that the entries in /etc/hosts (see fix below) file on both Master and Segments are correct. On the segment, the master host entry (used in a cluster) should be pointing to internal IP. Run a quick test of "telnet <hostname-on-/etc/hosts> <port-number> and check the ability to have a successful connection.
Rerun gpcheckcat or the failing query:
10.110.121.160 mdw-esx 172.28.19.220 mdw - reflects IP from ETH1 like below 172.28.19.228 sdw1 172.28.19.230 sdw2 172.28.19.229 sdw3 172.28.19.227 smdw
Note: /etc/hosts should be consistent across the clusters. If an external IP needs to be used then make sure the hostname used in the database for master should not point towards it.
Feedback
