Failed to start DB due to ERROR: could not access file ""plc_coordinator"": No such file or directory"

Products

• VMware Tanzu Greenplum

Issue/Introduction

gpstart failed due to the error message below in startup.log:

2021-03-04 12:20:39.862482 AEDT,,,p23963,th-202151808,,,,0,,,seg-1,,,,,"FATAL","58P01","could not access file ""plc_coordinator"": No such file or directory",,,,,,,,"internal_load_library","dfmgr.c",202,1 0xbf2adc postgres errstart (elog.c:557)
2 0xbf7acd postgres <symbol not found> (dfmgr.c:199)
3 0xbf84b4 postgres load_file (dfmgr.c:156)
4 0xc0b914 postgres process_shared_preload_libraries (miscinit.c:1378)
5 0xa12633 postgres PostmasterMain (postmaster.c:1151)
6 0x6b7b51 postgres main (main.c:205)
7 0x7f75f08e1555 libc.so.6 __libc_start_main + 0xf5
8 0x6c38ac postgres <symbol not found> + 0x6c38ac

Environment

Product Version: 6.13

Resolution

1. Since DB could not be started, we can check the GUC shared_preload_libraries in postgresql.conf. For example, it should be show like this:

# cat postgresql.conf | grep "shared_preload_libraries"
shared_preload_libraries="metrics_collector,plc_coordinator"

2. Check the version of PL/Container that is being used. Currently, we only need to add plc_coordinator into this GUC for the version 3 beta according to the documentation:

For PL/Container version 3 Beta only, add the plc_coordinator shared library to the Greenplum Database shared_preload_libraries server configuration parameter. 

3. If we are not using PL/Container 3 beta, we can backup postgresql.conf and then remove 'plc_coordinator' from the GUC "shared_preload_libraries". Do this for master, standby master, and primary/mirror segments which are failed to start due to the same error.

4. After that, DB should be able to start.