gpfdist fstream error - path escapes base directory
Article ID: 445303
Updated On:
Products
Issue/Introduction
When using gpfdist from Greenplum Client Tools version 7.x (or bundled with Greenplum 7.x), attempting to access files via symbolic links may fail with the following error in the gpfdist logs:
fstream: path escapes base directory: /path/to/target/file.dat
This typically occurs when the symbolic link is located within the gpfdist base directory, but points to a file located outside that directory tree.
Cause
This behavior is an intentional security enhancement in modern versions of gpfdist (built on Greenplum 7.x architecture). The fstream validation logic now enforces strict path resolution to prevent directory traversal.
gpfdist ensures that the resolved physical path of any requested file resides within the directory (the "base directory") where the gpfdist process was initiated. If a symlink resolves to a path outside this boundary, the request is rejected to prevent unauthorized access to system files.
Resolution
There is currently no configuration flag to bypass this security check. To resolve the issue, use one of the following methods:
1. Adjust the gpfdist Base Directory
Restart the gpfdist process from a higher-level directory that encompasses both the symbolic links and the actual target data files.
Example: If your files are in /data/files/ and your links are in /data/etl/links/, start gpfdist from a common parent directory like /data/ (ensure proper permissions are set).
2. Use Hard Links
If the target file is located on the same filesystem as the base directory, use hard links instead of symbolic links. Link the file with "ln <targetFile> <filename>" instead of "ln -s <targetFile> <filename>"
3. Relocate Data Files
Physically move or copy the target data files into the gpfdist base directory or one of its subdirectories so that no cross-directory traversal is required.
4. Feature Request
Broadcom R&D are developing an option to to bypass the restriction.
Additional Information
Client Tool Versions: While downgrading to Client Tools 7.4.1 or earlier may temporarily resolve the symptom (as older versions lacked this check), it is recommended to align Client Tools with the Greenplum Database version (e.g., 7.8.2) for long-term supportability and security.
Feedback
