admin@hcx-manager-hostname [ ~ ]$ systemctl status app-engine app-engine.service - App-Engine Loaded: loaded (/etc/systemd/system/app-engine.service; enabled; vendor preset: enabled) Active: activating (start-pre) since Thu <YYYY-MM-DD hh:mm:ss> UTC; 9min agoCntrl PID: 25616 (service-depende) Tasks: 2 Memory: 420.0K CGroup: /system.slice/app-engine.service ├─ 9572 sleep 30 └─25616 /bin/bash /etc/systemd/service-dependency-check.sh postgresdb database-upgrade zookeeper kafka<MMM DD hh:mm:ss> hcx-manager-hostname service-dependency-check.sh[25616]: kafka is not running.
ERROR Error while writing meta.properties to /common/kafka-db (org.apache.kafka.storage.internals.log.LogDirFailureChannel)java.nio.file.FileAlreadyExistsException: /common/kafka-db
[kafka-producer-network-thread | producer-TID_####-####-####-####, , , TxId: ] WARN o.apache.kafka.clients.NetworkClient- [Producer clientId=producer-TID_####-####-####-####, transactionalId=TID_####-####-####-####] Connection to node 0 (localhost/127.xxx.0.1:9092) could not be established. Broker may not be available.
VMware HCX
The disk expansion workflow for HCX 4.11.3 and above requires manual redirection of database directories from the /common partition to the newly expanded /common_ext partition. The failure occurs because the kafka-db and postgres-db directories were not correctly migrated or symlinked, preventing the services from accessing their data stores.
Also there are three more services which depends on Postgres and Kafka that needs to stopped before and started after applying the configuration changes.
Follow the steps mentioned in KB article for HCX manager in 4.11.3 and above - Increasing HCX Manager Disk Space for HCX Software Version 4.10 or Higher