VPXD service fails to start with "Peer authentication failed for user vpxd" error

Products

• VMware vCenter Server

Issue/Introduction

• The vmware-vpxd service fails to start or is forcibly shut down after multiple retries.
  
  
• The vSphere Client displays a "503 Service Unavailable" or "No healthy upstream" error.
  
  
• The issue frequently occurs following a vCenter FQDN change or certificate renewal.
  
  
• The /var/log/vmware/vpxd/vpxd.log, contains the following entries:
  
  YYYY-MM-DDTHH:MM:SS.265Z error vpxd[3098749] [Originator@6876 sub=vpxdVdb] [VpxdVdb::SetDBType] Failed to connect to database: ODBC error: (08001) - [unixODBC]connection to server on socket "/var/run/vpostgres/.s.PGSQL.5432" failed: FATAL:  Peer authentication failed for user "vpxd"
--> .  Retry attempt: 413057 ...
YYYY-MM-DDTHH:MM:SS.276Z info vpxd[3098749] [Originator@6876 sub=vpxdVdb] [VpxdVdb::SetDBType] Logging in to DSN: VMware VirtualCenter with username vpxd
YYYY-MM-DDTHH:MM:SS.280Z error vpxd[3098749] [Originator@6876 sub=vpxdVdb] [VpxdVdb::SetDBType] Aborting after 413058 retries.
  
  
• The /var/log/vmware/vpostgres/postgresql.log, contains the following entries:
  
  YYYY-MM-DDTHH:MM:SS.263 UTC 66cca993.b81c 0 [unknown] [unknown] [local] 47132 1LOG:  connection received: host=[local]
YYYY-MM-DDTHH:MM:SS.264 UTC 66cca993.b81c 0 VCDB vpxd [local] 47132 2LOG:  connection authenticated: identity="root" method=peer (/storage/db/vpostgres/pg_hba.conf:
24)
YYYY-MM-DDTHH:MM:SS.264 UTC 66cca993.b81c 0 VCDB vpxd [local] 47132 3LOG:  no match in usermap "vcdb" for user "vpxd" authenticated as "root"
YYYY-MM-DDTHH:MM:SS.264 UTC 66cca993.b81c 0 VCDB vpxd [local] 47132 4FATAL:  Peer authentication failed for user "vpxd"
YYYY-MM-DDTHH:MM:SS.264 UTC 66cca993.b81c 0 VCDB vpxd [local] 47132 5DETAIL:  Connection matched pg_hba.conf line 24: "local VCDB vpxd peer map=vcdb"

Environment

VMware vCenter Server 8.x

Cause

In vCenter Server 8.0, the vpxd process no longer uses standard database user credentials for its connection. Instead, it utilizes peer mapping, where the database authenticates the connection based on the Operating System (OS) user. This error occurs because the vpxd process is running as the OS user root rather than the expected OS user vpxd, causing the PostgreSQL database to reject the connection.

Resolution

There is no resolution at the moment. Follow these workaround steps to allow the root user to authenticate as the vpxd database user:

1. Log in to the vCenter Server Appliance as the root user via SSH.
   
   
   
2. Create a backup directory in root:
   
   mkdir /backup



3. Backup the following files to the newly created backup folder.
   

   cp /etc/vmware/vmware-vmon/svcCfgfiles/vpxd.json /backup/vpxd.json_backup

   cp /storage/db/vpostgres/pg_ident.conf /backup/pg_ident.conf_backup



4. Edit the vpxd.json file:

   vi /etc/vmware/vmware-vmon/svcCfgfiles/vpxd.json
   
   Locate the line "RunAsUser": "vpxd" and change it to "RunAsUser": "root". Save and exit.
   
   

5. Edit the pg_ident.conf file:

   vi /storage/db/vpostgres/pg_ident.conf

   Append the following line at the bottom:

   vcdb                root            vpxd

   Note: Use the Tab key twice to create the spacing between columns.
   
   

6. Update the database with this new configuration:

   psql -U postgres -d VCDB -c "SELECT pg_reload_conf();"



7. Restart vCenter Server services: 

   service-control --stop --all && service-control --start --all