"ERROR: connection with gpfdist failed for gpfdists:" error when running gpload in Tanzu Greenplum

Products

• VMware Tanzu Greenplum

Issue/Introduction

The gpload fails with the error "ERROR: connection with gpfdist failed for gpfdists:...".

ERROR: connection with gpfdist failed for gpfdists://gpload_server:8000//tmp/FF_SRC_1.txt. effective url: https://gpload_server:8000//tmp/FF_SRC_1.txt. (seg1 slice1 fd1f:a8d9:9f1c:8192::1a:6001 pid=31593) DETAIL: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca 

Environment

Product Version: 5.21

Resolution

To resolve this issue, you need to enable SSL or TLS with self-signed certificates in gpload. This article covers how to create and use self-signed certificates between Greenplum and gpload.

1. Generate the root certificate for your certificate authority.

openssl genrsa -des3 -out myCA.key 2048                                         // pass-phase at least 4 characters
openssl req -x509 -new -nodes -key myCA.key -sha256 -days 1825 -out myCA.pem    // use the pass-phase from step1 | Common Name : rootCA

​​​​​

2. Generate the key and cert for Greenplum segment servers.

openssl genrsa -out client.key 2048
openssl req -new -key client.key -out client.csr                                // Common Name : segments

vi client.ext
    authorityKeyIdentifier=keyid,issuer
    basicConstraints=CA:FALSE
    keyUsage = digitalSignature, nonRepudiation, keyEncipherment, dataEncipherment
    subjectAltName = @alt_names

    [alt_names]
    DNS.1 = segments

openssl x509 -req -in client.csr -CA myCA.pem -CAkey myCA.key -CAcreateserial -out client.crt -days 825 -sha256 -extfile client.ext // use the pass-phase from step1 

3. Generate the key and cert for a gpload server.

openssl genrsa -out server.key 2048
openssl req -new -key server.key -out server.csr                                 // Common Name : mdw_lab2
vi server.ext
    authorityKeyIdentifier=keyid,issuer
    basicConstraints=CA:FALSE
    keyUsage = digitalSignature, nonRepudiation, keyEncipherment, dataEncipherment
    subjectAltName = @alt_names
    
    [alt_names]
    DNS.1 = <your gpload hostname server here>

openssl x509 -req -in server.csr -CA myCA.pem -CAkey myCA.key -CAcreateserial -out server.crt -days 825 -sha256 -extfile server.ext // use the pass-phase from step1 

cp myCA.pem root.crt

4. To finalize this process, run this command:

- copy root.crt client.crt and client.key to all segment directories /gpfdists/ folder

5. Run a gpload.

gpload -f my_load.yml -V

The following is an example yaml file.

%YAML 1.1
---
VERSION: 1.0.0.1

DATABASE: mydb
USER: myuser
HOST: myhost
PORT: 5432
GPLOAD:
   INPUT:
     - SOURCE:
        FILE:
          - /tmp/FF_SRC_1.txt
        SSL: true
        CERTIFICATES_PATH: /usr/local/greenplum-loaders-5.26.0/ssl_certificates
     - COLUMNS:
       - "employee_id":
       - "badge":
     - MAX_LINE_LENGTH: 32768
     - FORMAT: TEXT
     - DELIMITER: "\x7c"
     - ESCAPE: '\'
     - NULL_AS: '\N'
     - QUOTE: '"'
     - ENCODING: 'utf8'
     - ERROR_LIMIT: 2
     - ERROR_TABLE:
   OUTPUT:
     - TABLE: public.employee
     - MODE: INSERT
   PRELOAD:
     - TRUNCATE: True

6. To verify the certificates, run this command:

[gpadmin@sdw1_lab2 gpfdists]$ openssl x509 -in client.crt  -text -noout | head -n 20
Certificate:
    Data:
        Version: 1 (0x0)
        Serial Number:
            ad:ef:39:02:9b:3a:51:59
    Signature Algorithm: sha256WithRSAEncryption
        Issuer: C=IN, ST=KA, L=BA, O=MYCOMPANY, OU=QA, CN=greenplum_segments/[email protected]
        Validity
            Not Before: May  7 07:32:17 2020 GMT
            Not After : May  5 07:32:17 2030 GMT
        Subject: C=IN, ST=KA, L=BA, O=MYCOMPANY, OU=QA, CN=greenplum_segments/[email protected]
        Subject Public Key Info:
            Public Key Algorithm: rsaEncryption
                Public-Key: (2048 bit)
                Modulus:
                    00:df:51:c6:63:e1:06:80:b2:8b:c9:d8:f2:e4:19: