hostd is not responding and cannot be killed by the kill -9 command

Products

• VMware vCenter Server
• VMware vSphere ESXi

Issue/Introduction

Symptoms:

• You cannot log in to an ESX host with VMware Infrastructure (VI) Client
• ESX host shows as disconnected in vCenter Server
• The hostd process is not responding
• You cannot kill hostd with the command kill -9 and the process keeps re-spawning itself

Environment

VMware vCenter Server 4.1.x
VMware vSphere ESXi 5.5
VMware ESXi 4.0.x Embedded
VMware vCenter Server Appliance 6.7.x
VMware ESXi 3.5.x Installable
VMware ESXi 4.0.x Installable
VMware vSphere ESXi 6.0
VMware ESX Server 3.0.x
VMware vCenter Server 5.0.x
VMware vCenter Server 6.7.x
VMware vCenter Server 6.5.x
VMware vCenter Server 5.5.x
VMware vSphere ESXi 5.1
VMware ESXi 3.5.x Embedded
VMware ESX 4.0.x
VMware vSphere ESXi 5.0
VMware ESX Server 3.5.x
VMware vCenter Server Appliance 6.0.x
VMware vCenter Server 4.0.x
VMware vCenter Server 6.0.x
VMware ESX 4.1.x
VMware vSphere ESXi 6.7
VMware ESXi 4.1.x Installable
VMware vSphere ESXi 6.5
VMware vCenter Server Appliance 6.5.x
VMware ESXi 4.1.x Embedded
VMware vCenter Server 5.1.x

Resolution

The vmware-watchdog process watches over hostd and restarts it if it detects that hostd is no longer running. This issue occurs if vmware-watchdog thinks that hostd is no longer running and keeps respawning hostd.

 

To resolve this issue, you must stop and restart hostd, so that vmware-watchdog stops respawning the process.

1. Run this command to stop hostd:
   
   service mgmt-vmware stop
   
   Note: In ESXi 5.x, the command is /etc/init.d/hostd stop.
    
2. Run this command to see if the vmware-watchdog thinks that hostd is stopped:
   
   Note: In ESXi hosts, the vmware-watchdog process command can be located at /opt/vmware/vpxa/bin/ and in ESXi 5.x, the watchdog.sh is located at /sbin. The following command can be run as /opt/vmware/vpxa/bin/vmware-watchdog -r hostd and this path also applies to the other subsequent vmware-watchdog commands.
   
   ./vmware-watchdog -r hostd
   
   For ESXi 5, a .sh extension is required to execute the command:
   
   /sbin/watchdog.sh -r hostd
    
   • If the command returns the message PID file not found, vmware-watchdog cannot see any hostd processes. Proceed to step 4.
   • If the vmware-watchdog command returns a PID file, run this command to kill hostd and proceed to step 3:
     
     vmware-watchdog -k hostd
      
   • If the vmware-watchdog command produces an error such as cannot find the PID, check to see if the PID file exists in the following location:
     
     /var/run/vmware/vmware-hostd.PID
     
     If this files exists, delete vmware-hostd.PID and watchdog-hostd.PID with the rm commands and proceed to step 3:
     
     rm /var/run/vmware/vmware-hostd.PID
     rm /var/run/vmware/watchdog-hostd.PID
      
3. Re-run this command:
   
   vmware-watchdog.sh -r hostd
    
4. Run this command to see if any hostd processes are running:
   
   ps | grep hostd
    
5. If no hostd processes are running, start the service. For more information, see Restarting the Management agents in ESXi

If these instructions do not resolve the issue, reboot the host.

Additional Information

To be alerted when this document is updated, click the Subscribe to Article link in the Actions box

Restarting the Management agents in ESXi