VMware HCX Manager services, specifically the Kafka and app-engine services, fail to initialize following a manual disk expansion of the HCX Manager appliance.
Symptoms:
Kafka service remains in a 'failed' or 'stopped' state.
From an SSH session to the HCX Manager, the app-engine service will be stuck in activating state since kafka is not running:admin@hcx-manager-hostname [ ~ ]$ systemctl status app-engine app-engine.service - App-Engine Loaded: loaded (/etc/systemd/system/app-engine.service; enabled; vendor preset: enabled) Active: activating (start-pre) since Thu <YYYY-MM-DD hh:mm:ss> UTC; 9min agoCntrl PID: 25616 (service-depende) Tasks: 2 Memory: 420.0K CGroup: /system.slice/app-engine.service ├─ 9572 sleep 30 └─25616 /bin/bash /etc/systemd/service-dependency-check.sh postgresdb database-upgrade zookeeper kafka
<MMM DD hh:mm:ss> hcx-manager-hostname service-dependency-check.sh[25616]: kafka is not running.<MMM DD hh:mm:ss> hcx-manager-hostname service-dependency-check.sh[25616]: kafka is not running.<MMM DD hh:mm:ss> hcx-manager-hostname service-dependency-check.sh[25616]: kafka is not running.<MMM DD hh:mm:ss> hcx-manager-hostname service-dependency-check.sh[25616]: kafka is not running.<MMM DD hh:mm:ss> hcx-manager-hostname service-dependency-check.sh[25616]: kafka is not running.<MMM DD hh:mm:ss> hcx-manager-hostname service-dependency-check.sh[25616]: kafka is not running.
Logs may indicate IO errors or missing directory paths.
Filesystem inspection reveals the /common/kafka-db path is inconsistent with the expected post-resize structure.
VMware HCX 4.10.x, 4.11.x
The disk expansion workflow for HCX 4.10+ requires manual redirection of database directories from the /common partition to the newly expanded /common_ext partition. The failure occurs because the kafka-db and postgres-db directories were not correctly migrated or symlinked, preventing the services from accessing their data stores.
Use KB Increasing HCX Manager Disk Space for HCX Software Version 4.10 or Higher on a HCX manager that are 4. 10 and greater.