/var/log/vmware/vmon/vmon.log
will show similar entries to: YYYY-MM-DD-THH:MM:SS In(05) host-XXXX Received start request for vmware-vpostgres
YYYY-MM-DD-THH:MM:SS In(05) host-XXXX <vmware-vpostgres-prestart> Constructed command: /opt/vmware/vpostgres/current/scripts/pg_pre_start
YYYY-MM-DD-THH:MM:SS Wa(03) host-XXXX <vmware-vpostgres> Service pre-start command's stderr: Issuing signal KILL on all PostgreSQL processes owned by OS user vpostgres
YYYY-MM-DD-THH:MM:SS Wa(03)+ host-XXXX Traceback (most recent call last):
YYYY-MM-DD-THH:MM:SS Wa(03)+ host-XXXX File "/opt/vmware/vpostgres/13/share/python-modules/vpostgres_cis/telemetry.py", line 247, in pushdata
YYYY-MM-DD-THH:MM:SS Wa(03)+ host-XXXX response = urllib.request.urlopen(request)
YYYY-MM-DD-THH:MM:SS Wa(03)+ host-XXXX File "/usr/lib/python3.7/urllib/request.py", line 222, in urlopen
YYYY-MM-DD-THH:MM:SS Wa(03)+ host-XXXX return opener.open(url, data, timeout)
YYYY-MM-DD-THH:MM:SS Wa(03)+ host-XXXX File "/usr/lib/python3.7/urllib/request.py", line 531, in open
YYYY-MM-DD-THH:MM:SS Wa(03)+ host-XXXX response = meth(req, response)
YYYY-MM-DD-THH:MM:SS Wa(03)+ host-XXXX File "/usr/lib/python3.7/urllib/request.py", line 641, in http_response
YYYY-MM-DD-THH:MM:SS Wa(03)+ host-XXXX 'http', request, response, code, msg, hdrs)
YYYY-MM-DD-THH:MM:SS Wa(03)+ host-XXXX File "/usr/lib/python3.7/urllib/request.py", line 569, in error
YYYY-MM-DD-THH:MM:SS Wa(03)+ host-XXXX return self._call_chain(*args)
YYYY-MM-DD-THH:MM:SS Wa(03)+ host-XXXX File "/usr/lib/python3.7/urllib/request.py", line 503, in _call_chain
YYYY-MM-DD-THH:MM:SS Wa(03)+ host-XXXX result = func(*args)
YYYY-MM-DD-THH:MM:SS Wa(03)+ host-XXXX File "/usr/lib/python3.7/urllib/request.py", line 649, in http_error_default
YYYY-MM-DD-THH:MM:SS Wa(03)+ host-XXXX raise HTTPError(req.full_url, code, msg, hdrs, fp)
YYYY-MM-DD-THH:MM:SS Wa(03)+ host-XXXX urllib.error.HTTPError: HTTP Error 503: Service Unavailable
YYYY-MM-DD-THH:MM:SS Wa(03) host-XXXX <vmware-vpostgres> Service pre-start command's stderr: yyyy-mm-ddThh:mm:ss.153Z Failed to push telemetry data for startup
YYYY-MM-DD-THH:MM:SS Wa(03)+ host-XXXX
YYYY-MM-DD-THH:MM:SS Wa(03) host-XXXX <vmware-vpostgres> Service pre-start command's stderr: Could not generate telemetry data
YYYY-MM-DD-THH:MM:SS In(05) host-XXXX <vmware-vpostgres> Service pre-start command completed successfully.
YYYY-MM-DD-THH:MM:SS In(05) host-XXXX <vmware-vpostgres> Constructed command: /opt/vmware/vpostgres/current/bin/postgres -D /storage/db/vpostgres
YYYY-MM-DD-THH:MM:SS host-XXXX Client info Uid=0,Gid=0,Pid=34887,Comm=(vmon-coredumper),PPid=2,Comm=(kthreadd),PPid=0
YYYY-MM-DD-THH:MM:SS Wa(03) host-XXXX [ReadSvcSubStartupData] No startup information from vmware-vpostgres.
YYYY-MM-DD-THH:MM:SS Wa(03) host-XXXX <vmware-vpostgres> Service exited. Exit code 1
/var/log/vmware/vpostgres/postgresql-xx.log
will show similar entries to: YYYY-MM-DD-THH:MM:SS 67a573d7.26bf 0 LOG: listening on IPv6 address "::", port 5432
YYYY-MM-DD-THH:MM:SS 67a573d7.26bf 0 LOG: listening on Unix socket "/var/run/vpostgres/.s.PGSQL.5432"
YYYY-MM-DD-THH:MM:SS 67a573d7.26c1 0 LOG: database system was interrupted; last known up at 2024-09-20 03:07:24 UTC
YYYY-MM-DD-THH:MM:SS 67a573d7.26c1 0 PANIC: replication slot file "pg_replslot/vpg_archiver/state" has wrong magic number: 875771188 instead of 17112225
YYYY-MM-DD-THH:MM:SS 67a573d7.26bf 0 LOG: startup process (PID 9921) was terminated by signal 6: Aborted
YYYY-MM-DD-THH:MM:SS 67a573d7.26bf 0 LOG: aborting startup due to startup process failure
YYYY-MM-DD-THH:MM:SS 67a573d7.26bf 0 LOG: database system is shut down
vCenter Server 7.X
vCenter Server 8.X
This issue occurs due to the corrupt pg_replslot/vpg_archiver/state
file.
Before proceeding with the steps below, take both backup and a snapshot of the vCenter Server Appliance. If the vCenter is part of a Enhanced Linked Mode (ELM) replication setup, also take a backup or offline (powered off) snapshot of all replicating vCenter ELM nodes.
/storage/db/vpostgres/pg_replslot
rm -r vpg_archiver
service-control --start vmware-vpostgres
Note: The state file gets re-created as we delete everything from directory "vpg_archiver" which is refreshed (recreated) when we issue the command to start the vpostgres service.