gpload - http response code 400 from gpfdist (....) HTTP/1.0 400 path too long

Products

• VMware Tanzu Greenplum

Issue/Introduction

Received the following error when running gpload:

"ERROR","08006","http response code 400 from gpfdist (gpfdist://host01:8000//opt/output/location/appl/nydnkrds/new_dir/data/local/prod/out/MyFiles/marked_order_ny_locally_mmq_hrly.out%20/opt/output/location/appl/nydnkrds/new_dir/data/local/prod/out/MyFiles/marked_order_ny_locally_etpmm_hrly.out%20/opt/output/location/appl/nydnkrds/new_dir/data/local/prod/out/MyFiles/marked_order_ny_locally_ny1_hrly.out%20/opt/output/location/appl/nydnkrds/new_dir/data/local/prod/out/MyFiles/marked_order_ny_locally_rfq_hrly.out%20/opt/output/location/appl/nydnkrds/new_dir/data/local/prod/out/MyFiles/marked_order_ny_locally__hrly.out%20/opt/output/location/appl/nydnkrds/new_dir/data/local/prod/out/MyFiles/marked_order_ny_locally_nys_hrly.out%20/opt/output/location/appl/nydnkrds/new_dir/data/local/prod/out/MyFiles/marked_order_ny_locally_hrly.out%20/opt/output/location/appl/nydnkrds/new_dir/data/local/prod/out/MyFiles/marked_order_ny_locally_hrly.out%20/opt/output/location/appl/nydnkrds/new_dir/data/local/prod/out/MyFiles/marked_order_ny_locally_1_hrly.out%20/opt/output/location/appl/nydnkrds/new_dir/data/local/prod/out/MyFiles/marked_order_ny_locally_2_hrly.out): HTTP/1.0 400 path too long (seg66 slice1 10.10.10.6:16728 pid=196260)"

This error can occur when there are many files specified in the YAML file for gpload. The concatenation of the path and file names cannot exceed 1023 characters.

Environment

Product Version: 5.27

Resolution

This is a limitation set in the code of gpfdist. To avoid this issue either use wildcards to specify the files or limit the number of files specified. So instead of listing the file individually as:

 FILE
      - 

/opt/output/location/appl/nydnkrds/new_dir/data/local/prod/out/MyFiles/marked_order_ny_locally_mmq_hrly.out - /opt/output/location/appl/nydnkrds/new_dir/data/local/prod/out/MyFiles/marked_order_ny_locally_etpmm_hrly.out - /opt/... List the file with a wildcard:

   FILE
      - /opt/output/location/appl/nydnkrds/new_dir/data/local/prod/out/MyFiles/*