vPostgres service fails to start on vCenter Server Appliance Error : FATAL: lock file "/var/run/vpostgres/.s.PGSQL.5432.lock" is empty

Products

• VMware vCenter Server

Issue/Introduction

Symptoms:

• vCenter services will fail to start as vPostgres service is in a stopped state
• When try to start the service manually we get the following error trace :

Operation not cancellable. Please wait for it to finish...
Performing start operation on service vmware-vpostgres...
Error executing start on service vmware-vpostgres. Details {
    "detail": [
        {
            "args": [
                "vmware-vpostgres"
            ],
            "id": "install.ciscommon.service.failstart",
            "translatable": "An error occurred while starting service '%(0)s'",
            "localized": "An error occurred while starting service 'vmware-vpostgres'"
        }
    ],
    "componentKey": null,
    "resolution": null,
    "problemId": null
}
Service-control failed. Error: {
    "detail": [
        {
            "args": [
                "vmware-vpostgres"
            ],
            "id": "install.ciscommon.service.failstart",
            "translatable": "An error occurred while starting service '%(0)s'",
            "localized": "An error occurred while starting service 'vmware-vpostgres'"
        }
    ],
    "componentKey": null,
    "resolution": null,
    "problemId": null
}

Environment

VMware vCenter Server Appliance 6.7.x

Cause

• Checking the serverlog.stderr log we find the following:

Starting service process with pid: 17384.
LOG:  skipping missing configuration file "/storage/db/vpostgres/postgresql.conf.repl"
LOG:  skipping missing configuration file "/storage/db/vpostgres/postgresql.conf.repl"
2019-11-21 14:52:54.932 UTC 5dd6a4c6.43e8 0   FATAL:  lock file "/var/run/vpostgres/.s.PGSQL.5432.lock" is empty
2019-11-21 14:52:54.932 UTC 5dd6a4c6.43e8 0   HINT:  Either another server is starting, or the lock file is the remnant of a previous server startup crash.
2019-11-21 14:52:54.932 UTC 5dd6a4c6.43e8 0   LOG:  database system is shut down
 

• The vPostgres Service is failing to start because the contents of ".s.PGSQL.5432.lock" database lock file are missing.

Resolution

Follow the below steps to solve this issue:

1. Take a Snapshot of the vCenter Server Appliance virtual machine.
2. Login to the Appliance shell via SSH.
3. Got to this location : /var/run/vpostgres
4. Locate the file named as .s.PGSQL.5432.lock
5. Open the file in editor : vi .s.PGSQL.5432.lock
6. Enter the following in the file:

21810
/storage/db/vpostgres
1574278293
5432
/var/run/vpostgres
.s.PGSQL.5432.lock (END)

7. Save the file and restart all the vCenter Services.

Additional Information

Impact/Risks:
vCenter services including the vpxd won't start until and unless vPostgres service is running.