psql with ldap authentication intermittently hang.

Products

• VMware Tanzu Data Suite

Issue/Introduction

When use ldap authentication, if met psql connection intermittently hang.  From pg log, after set log level to debug1, it may report below message:

could not search LDAP for filter  "ldap_server_location"  : operation error.
"FATAL","57014","canceling authentication due to timeout"

Environment

GPDB 7.x, 6.x

Cause

There are many reasons can cause this issue, here is one possible situation.

1. The ldap server in pg_hba.conf entry point to a load balancer server. This load balancer server will redirect the ldap authentication request to the actual ldap server.
2. There are multiple ldap server in place, the load balance can redirect to any ldap server. Each ldap server may has subdomain server. 
3. If any ldap server is unstable or any subdomain server has connection issue with greenplum master host, then the redirected ldap authentication request will be in waiting status, then from gpdb side, the psql connection will also be in waiting status.

Resolution

For trouble shooting purpose, we can update the pg_hba.conf file, use the actual ldap server instead of the load balancer server, then test connection for a few times. We can also disable the referral function in LDAP server (default is on).

in master host's LDAP configuration file (/etc/openldap/ldap.conf), add below setting:
REFERRALS off

After this change, it's necessary to reload gpdb configuration (gpstop -u).