HCX Services Not Starting After Upgrade

Products

• VMware HCX

Issue/Introduction

After completing an HCX upgrade, the HCX services may fail to start. Although the upgrade process succeeds, the system may appear unresponsive or incomplete.

• Checking ownership of services:

• • Login HCX manager as admin.
  • $ cd /common
  • List services using ls -ltrh and check for postgres service ownership.

• Any one or all of the service permissions for the directories under /common may appear as follows::
  • appliance-management: "root hadoop"
  • kafka-db:                         "ldap docker"
  • postgres-db:                    "systemd-coredump hadoop"
  • zookeeper-db:                 "999 users"

 

 

 

 

Environment

HCX

Cause

If postgres service ownership is incorrectly assigned to the systemd-coredump group instead of the expected postgres user and group. Postgres service fails to start.

Issue:

drwx------  19 systemd-coredump hadoop  4.0K ## ## 00:00 postgres-db

Working/Expected:

drwx------  19 postgres        postgres  4.0K ## ## 00:00 postgres-db

 

This issue occurs when the HCX appliance is manually rebooted during the critical initialization steps after the upgrade bundle is uploaded and extracted.

Resolution

Avoid manual rebooting during the HCX manager upgrade process.

The HCX upgrade process includes:

1. Upload and extraction of the HCX upgrade bundle.
2. System reboot.
3. Execution of the first boot script.
4. Second reboot.
5. Execution of upgrade scripts.

Total upgrade duration: 30–45 minutes (including 2 automatic reboots).

During the initial boot phase, it is expected that the Postgres service does not start. Interrupting this process with a manual reboot can cause permission inconsistencies—such as incorrect ownership of HCX-related services—and prevent proper initialization.

Additional Information

• Do not interrupt the HCX upgrade by manually rebooting unless instructed by VMware support.