Service mgmt-vmware restart may not restart hostd in ESX/ESXi

Products

• VMware vSphere ESXi

Issue/Introduction

Symptoms:

• The command service mgmt-vmware restart does not restart hostd.
• The script gets stuck when stopping the service.
• The SSH session to the ESX/ESXi host becomes unresponsive.
• hostd fails to restart.
• Neither of these commands restart hostd successfully on an ESXi/ESX host:

• • ./sbin/services.sh
  • ./etc/init.d/hostd restart

Environment

VMware ESXi 4.0.x Embedded
VMware vSphere ESXi 5.0
VMware vSphere ESXi 5.1
VMware ESXi 4.1.x Embedded
VMware ESX Server 3.0.x
VMware vSphere ESXi 5.5
VMware ESXi 4.1.x Installable
VMware vSphere ESXi 6.5
VMware ESX 4.1.x
VMware ESX 4.0.x
VMware ESXi 3.5.x Embedded
VMware vSphere ESXi 6.0
VMware ESX Server 3.5.x
VMware vSphere ESXi 6.7
VMware ESXi 4.0.x Installable
VMware ESXi 3.5.x Installable

Cause

This issue occurs if the config.xml file, located at /etc/vmware/hostd/ and the proxy.xml file, located at /etc/vmware/hostd/ is empty or has invalid entries. This issue may also occur if there are IPv6 entries in the /etc/hosts file that are associated with the localhost.

Resolution

To resolve this issue, manually stop and restart the stuck service.

 

To stop and restart the stuck service:

 

1. Log in as root to the ESXi/ESX host command-line via the physical console or via KVM connection.
2. Navigate to the /var/run/vmware directory with the command:
   
   # cd /var/run/vmware
    
3. Run this command to list the files vmware-hostd.PID and watchdog-hostd.PID:
   
   # ls -l vmware-hostd.PID watchdog-hostd.PID
    
4. Determine theProcess ID (PID) management service. View the contents of the vmware-hostd.PID file with the command:
   
   # cat vmware-hostd.PID
   
   For example:
   
   [root@vmware]# cat vmware-hostd.PID
   1191[root@vmware]#
    
5. Use the resulting PID to kill the process.
   
   Caution: Use the kill -9 command with care. It kills the process of the supplied PID without exception or confirmation.
   
   # kill -9 PID
   
   For example:
   
   kill -9 1191
    
6. Delete the vmware-hostd.PID and watchdog-hostd.PID files with the command:
   
   # rm vmware-hostd.PID watchdog-hostd.PID
    
7. Start the management service:
    
   • For ESX, run the command:
     
     # service mgmt-vmware start
      
   • For ESXi, run the command:
     
     # /etc/init.d/hostd start

Notes:

• This issue may also occur if the /etc/vmware/hostd/config.xml is blank or has invalid entries. If a backup of the config.xml file exists in the same directory, the issue may be resolved by replacing the config.xml file with this backup file.
• This issue also occurs if the /etc/vmware/hostd/proxy.xml file is empty or has invalid entries. This issue may be resolved by copying the proxy.xml file from a working host and then restarting the hostd service.

Additional Information

Note: In the hostd.log file, when the proxy.xml file is empty, you see the messages similar to:

[2012-07-30 04:13:26.764 FFFCAE80 verbose 'App'] Plugin 9 statically linked
[2012-07-30 04:13:26.764 FFFCAE80 info 'Proxysvc'] Proxy config filepath is /etc/vmware/hostd/proxy.xml
[2012-07-30 04:13:26.765 FFFCAE80 panic 'App'] error: no element found
[2012-07-30 04:13:26.765 FFFCAE80 panic 'App'] backtrace:
[00] rip 16d00c33
[01] rip 16b9303e
[02] rip 16b327a2

 

 

Restarting the Management agents in ESXi