The mgmt-vmware service is the service that starts hostd. Instead of starting hostd directly, mgmt-vmware runs a script called vmware-watchdog which in turn starts hostd as a child process. After vmware-watchdog starts hostd, it waits for (monitors) it and restarts hostd if hostd fails or generates debug/support info after a number of failures.
When mgmt-vmware starts successfully and hostd is running normally, ps shows the following processes:
The watchdog (parent process):
/bin/sh /usr/bin/vmware-watchdog -s hostd -u 60 -q 5 -c /usr/sbin/vmware-hostd-support /usr/sbin/vmware-hostd -u
The hostd (child process):
/usr/lib/vmware/hostd/vmware-hostd /etc/vmware/hostd/config.xml -u
Before mgmt-vmware tells vmware-watchdog to start hostd, it first tells vmware-watchdog to check if hostd is already running. mgmt-vmware does not tell vmware-watchdog to start hostd if vmware-watchdog reports that hostd is already running.
vmware-watchdog performs the following checks to see if hostd is already running:
- Is there something in the /var/run/vmware/watchdog-hostd.PID file? When vmware-watchdog starts hostd, it writes the PID of the watchdog process to this file.
- Is there a vmware-watchdog hostd monitoring process already running? It checks the ps output.
When mgmt-vmware service or hostd is stopped, vmware-watchdog deletes the watchdog-hostd.PID file and exits. Sometimes, if hostd or vmware-watchdog fails to exit cleanly, hostd terminates but vmware-watchdog may still be running or may not delete the watchdog-hostd.PID, therefore preventing the mgmt-vmware service from starting again.
Symptoms:
- The mgmt-vmware service does not start because it incorrectly thinks that the Host Agent (hostd) is already running.
- The following dialog appears when trying to start mgmt-vmware:
VMware ESX Server Host Agent is already running
- Listing the running processes (ps aux) shows that hostd is not running.