ERROR: could not load library "/some/custom/install/path/16/lib64/postgres_fdw.so"

Products

• VMware Tanzu Data Suite
• VMware Tanzu PostgreSQL

Issue/Introduction

pg_ctl fails with the below or similar error:

                              List of installed extensions
     Name     | Version |   Schema   |                    Description
--------------+---------+------------+----------------------------------------------------
 file_fdw     | 1.0     | mspsg      | foreign-data wrapper for flat file access
 plpgsql      | 1.0     | pg_catalog | PL/pgSQL procedural language
 plpython3u   | 1.0     | pg_catalog | PL/Python3U untrusted procedural language
 postgres_fdw | 1.1     | mspsgext   | foreign-data wrapper for remote PostgreSQL servers
(4 rows)

could not load library "$libdir/postgres_fdw": ERROR:  could not load library "/custom_install_path/vmware/postgres/16/lib64/postgres_fdw.so": /custom_install_path/vmware/postgres/16/lib64/postgres_fdw.so: undefined symbol: PQconnectionUsedGSSAPI
In database: databaseX

Environment

Tanzu for Postgress 16.6.0

Cause

a dynamically linked library is missing from a library directory in the library search path. This can happen if a custom install directory is used, or the original install path was changed after installation

Resolution

1.] Verify that the issue is a missing library using the 'ldd' bash native:

[root@postgreshost ~]# ldd /custom_install_path/vmware/postgres/16/lib64/postgres_fdw.so 
	linux-vdso.so.1 (0x00007fffd7af5000)
	libpq.so.5 => not found
	libc.so.6 => /lib64/libc.so.6 (0x00007ff10a1c6000)
	/lib64/ld-linux-x86-64.so.2 (0x00007ff10a7bc000)

2.] From the above output we see that a postgres_fdw.so required library (libpq.so.5) is missing (not found) from the dynamic library directories on the current system. In this case, libpq.so.5 comes packaged with Tanzu for Postgress 16.6.0. Verify that the artifact exists within the postgres library directory:

[root@postgreshost ~]$ ls /custom_install_path/vmware/postgres/16/lib64/libpq.so.5
/custom_install_path/vmware/postgres/16/lib64/libpq.so.5

 

3.] Since we see that the required library artifact is present, update the LD_LIBRARY_PATH environment variable so that it points to the Tanzu for Postgress install library directory:

export LD_LIBRARY_PATH=/custom_install_path/vmware/postgres/16/lib64

 

3.] After the copy, verify once more that postgres_fdw.so has access to all required libraries:

[root@postgreshost ~]# ldd /custom_install_path/vmware/postgres/16/lib64/postgres_fdw.so
	linux-vdso.so.1 (0x00007fff80bc7000)
	libpq.so.5 => /lib64/libpq.so.5 (0x00007f455a465000)
	libc.so.6 => /lib64/libc.so.6 (0x00007f455a08f000)
	libssl.so.1.1 => /lib64/libssl.so.1.1 (0x00007f4559dfa000)
	libcrypto.so.1.1 => /lib64/libcrypto.so.1.1 (0x00007f455990f000)
	libgssapi_krb5.so.2 => /lib64/libgssapi_krb5.so.2 (0x00007f45596ba000)
	libldap_r-2.4.so.2 => /lib64/libldap_r-2.4.so.2 (0x00007f4559462000)
	libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f4559242000)
	/lib64/ld-linux-x86-64.so.2 (0x00007f455a8d9000)
	libz.so.1 => /lib64/libz.so.1 (0x00007f455902a000)
	libdl.so.2 => /lib64/libdl.so.2 (0x00007f4558e26000)
	libkrb5.so.3 => /lib64/libkrb5.so.3 (0x00007f4558b3b000)
	libk5crypto.so.3 => /lib64/libk5crypto.so.3 (0x00007f4558924000)
	libcom_err.so.2 => /lib64/libcom_err.so.2 (0x00007f4558720000)
	libkrb5support.so.0 => /lib64/libkrb5support.so.0 (0x00007f455850f000)
	libkeyutils.so.1 => /lib64/libkeyutils.so.1 (0x00007f455830b000)
	libresolv.so.2 => /lib64/libresolv.so.2 (0x00007f45580f3000)
	liblber-2.4.so.2 => /lib64/liblber-2.4.so.2 (0x00007f4557ee3000)
	libsasl2.so.3 => /lib64/libsasl2.so.3 (0x00007f4557cc5000)
	libselinux.so.1 => /lib64/libselinux.so.1 (0x00007f4557a9a000)
	libcrypt.so.1 => /lib64/libcrypt.so.1 (0x00007f4557871000)
	libpcre2-8.so.0 => /lib64/libpcre2-8.so.0 (0x00007f45575ed000)

4.] retry pg_ctl