Linux OS command not working after loaded greenplum_path.sh file

Products

• VMware Tanzu Greenplum

Issue/Introduction

When we upgrade the operation system to a certain release of RHEL or RHEL-like OS (for example 8.9+), If we load(# source) the greenplum_path.sh file, we might notice that some OS commands will no longer work. 

For example: 

# source /usr/local/greenplum-db-6.22.0/greenplum_path.sh
# rpm 
rpm: symbol lookup error: /lib64/libelf.so.1: undefined symbol: ZSTD_compressStream2
# ip
ip: symbol lookup error: /lib64/libelf.so.1: undefined symbol: ZSTD_compressStream2

The reason is because greenplum_path.sh file will set the OS environment parameter $LD_LIBRARY_PATH, with the path under $GPHOME

echo $LD_LIBRARY_PATH
/usr/local/greenplum-db-6.22.0/lib:/usr/local/greenplum-db-6.22.0/ext/python/lib:/opt/greenplum_6.22.0/lib:/opt/greenplum_6.22.0/ext/python/lib:/opt/greenplum_6.22.0/ext/python3.9/lib:/usr/local/greenplum-db-6.22.0/ext/python3.9/lib

The libzstd.so under $GPHOME conflicts with the OS lib file: the other OS lib file needs to use function ZSTD_compressStream2 but the libzstd in $GPHOME does not have such a function

See below for example:

### The OS libzstd lib file: 

# ll /lib64/libzstd.so.1
lrwxrwxrwx. 1 root root 16 Apr 12  2021 /lib64/libzstd.so.1 -> libzstd.so.1.4.4

# readelf -s /lib64/libzstd.so.1 | grep ZSTD_compressStream2
    61: 0000000000011c70   131 FUNC    GLOBAL DEFAULT   12 ZSTD_compressStream2_simp
   181: 0000000000011390  2210 FUNC    GLOBAL DEFAULT   12 ZSTD_compressStream2

### the Greenplum provided libzstd lib file: 

# ll $GPHOME/lib/libzstd.so.1
lrwxrwxrwx 1 gpadmin gpadmin 16 Sep 10  2022 /opt/greenplum_6.22.0/lib/libzstd.so.1 -> libzstd.so.1.3.7

# readelf -s $GPHOME/lib/libzstd.so.1 | grep ZSTD_compressStream2
<no output>

Environment

Product Version: 6.25

Resolution

Workaround:

We need to handle 2 kinds of scenarios:

1. Run the OS command from the master host only.

LD_LIBRARY_PATH= rpm --version
RPM version 4.14.3

2. Run some tools like gppkg, which will run the command on the remote host after "source" the greenplum_path.sh  

a. backup greenplum_path.sh then update greenplum_path.sh under $GPHOME, add 1 line at the end 

export LD_LIBRARY_PATH=

b. copy the file to all the segment hosts
c. run gppkg or other commands we need
d. Once step #c is finished, revert the greenplum_path.sh file to the original one. 

The Fix:

The issue has been fixed under 6.26.1 and afterward, see the release note for more details
https://docs.vmware.com/en/VMware-Greenplum/6/greenplum-database/relnotes-release-notes.html

Zstandard (zstd) Supported Version
This release updates the supported zstd version on RHEL 8 to 1.4.4.