vCSA 7 patch/upgrade failed for vtsdb "Exception('Inplace pg_upgrade failed')"
search cancel

vCSA 7 patch/upgrade failed for vtsdb "Exception('Inplace pg_upgrade failed')"


Article ID: 313300


Updated On:


VMware vCenter Server


vCenter 7 patch/upgrade failed with "Exception('Inplace pg_upgrade failed').

2022-06-04 02:01:25,190.190Z vtsdb:Patch ERROR vmware_b2b.patching.executor.hook_executor Patch hook 'vtsdb:Patch' failed.
Traceback (most recent call last):
  File "/storage/updatemgr/software-updatew0jceyl0/stage/scripts/patches/py/vmware_b2b/patching/executor/", line 74, in executeHook
    executionResult = systemExtension(args)
  File "/storage/updatemgr/software-updatew0jceyl0/stage/scripts/patches/libs/sdk/", line 106, in __call__
    result = self.extension(*args)
  File "/storage/updatemgr/software-updatew0jceyl0/stage/scripts/patches/libs/sdk/", line 123, in _func
    return func(*args)
  File "/storage/updatemgr/software-updatew0jceyl0/stage/scripts/patches/payload/components-script/vtsdb/", line 144, in patch
    raise Exception('Inplace pg_upgrade failed')
Exception: Inplace pg_upgrade failed
2022-06-04T02:01:25.195Z ERROR vmware_b2b.patching.phases.patcher Patch hook Patch got ComponentWrapperError.
Traceback (most recent call last):
  File "/storage/updatemgr/software-updatew0jceyl0/stage/scripts/patches/py/vmware_b2b/patching/phases/", line 203, in patch
    _patchComponents(ctx, userData, statusAggregator.reportingQueue)
  File "/storage/updatemgr/software-updatew0jceyl0/stage/scripts/patches/py/vmware_b2b/patching/phases/", line 85, in _patchComponents
    executeComponentHook(Hook.Patch, ctx, c, userData, reportingQueue)
  File "/storage/updatemgr/software-updatew0jceyl0/stage/scripts/patches/py/vmware_b2b/patching/executor/", line 98, in executeComponentHook
    reportQueue, identifier, expectedResultType)
  File "/storage/updatemgr/software-updatew0jceyl0/stage/scripts/patches/py/vmware_b2b/patching/executor/", line 53, in executeHook
    result = executor.executeHook(scriptFile, hook, args, reportQueue, reportIdentifier)
  File "/storage/updatemgr/software-updatew0jceyl0/stage/scripts/patches/py/vmware_b2b/patching/executor/", line 119, in executeHook
    raise ex
2022-06-04T02:01:26.197Z WARNING root stopping status aggregation...
2022-06-04T02:01:26.197Z ERROR __main__ Patch vCSA failed


Performing Consistency Checks on Old Live Server
Checking cluster versions ok
connection to database failed: could not connect to server: Connection refused
        Is the server running locally and accepting
        connections on Unix domain socket "/var/run/vtsdb/.s.PGSQL.5433"?


command: "/opt/vmware/vpostgres/current/../12/bin/pg_ctl" -w -l "pg_upgrade_server.log" -D "/storage/vtsdb/vtsdbdata" -o "-p 50432 -c autovacuum=off -c autovacuum_freeze_max_age=2000000000 -c listen_addresses='' -c unix_socket_permissions=0700" start >> "pg_upgrade_server.log" 2>&1
pg_ctl: another server might be running; trying to start server anyway
waiting for server to start....2022-06-04 02:01:25.096 UTC 629abcf5.74fe 0 FATAL: pre-existing shared memory block (key 5433001, ID 32769) is still in use
2022-06-04 02:01:25.096 UTC 629abcf5.74fe 0 HINT: Terminate any old server processes associated with data directory "/storage/vtsdb/vtsdbdata".
 stopped waiting
pg_ctl: could not start server


VMware vCenter Server 7.0.x


vmon was issuing SIGTERM StopSignal(as default) to shutdown vtsdb during the upgrade, which was leading to a force kill of vtsdb as a result of shutdown process timeout.


This issue is fixed in vCenter version 7.0 U3i.

In the fix provided, we are explicitly issuing SIGINT as StopSignal, which leads to fast shutdown, as compared to smart shutdown through SIGTERM which was issued earlier by default. We have also specified StopBefore param to shutdown vstats before vtsdb.
Smart Shutdown vs Fast Shutdown: