GPSS job name truncation causes staging table collisions

Products

• VMware Tanzu Greenplum
• VMware Tanzu Greenplum / Gemfire

Issue/Introduction

In GPSS, jobs with long names (typically exceeding Greenplum/PostgreSQL’s 63-character identifier limit) can result in staging table name collisions.

When two GPSS jobs have similar long names (for example):

gpss_test_rabbitmq_to_greenplum_job_with_long_name01
gpss_test_rabbitmq_to_greenplum_job_with_long_name02

 

GPSS internally constructs staging tables with a fixed prefix (for example, gprabbitmqloadext_) followed by the job name. Since Greenplum and PostgreSQL limit identifiers to 63 characters, the resulting table name is silently truncated and both jobs may reference the same external table name, for example:

"public"."gprabbitmqloadext_pss_test_rabbitmq_to_greenplum_job_with_long"

 

This table name truncation causes multiple distinct jobs to reference the same staging table, resulting in:

• Intermittent job failures
  
  
• Errors such as "relation does not exist" and "gpfdist 404 Not Found"

Resolution

Workaround:

Shorten the file name for the job to reduce the length of the job name.

Keep the total length of the the table name (prefix + job_name) to 63 characters or less.

Fix:

It is planned to improve the behaviour in GPSS 2.3.0.