ESXi 5.0 reports multiple issues with syslog configuration

Products

• VMware vCenter Server
• VMware vSphere ESXi

Issue/Introduction

Symptoms:

On ESXi 5.0, you experience these symptoms:

Logging:

• ESXi 5.0 has stopped logging.
• The /scratch/log location contains only the old log files.
• The /var/log/ directory is missing key log files including auth.log , fdm.log , hostd.log , syslog.log , vmkernel.log , vmksummary.log , and vmkwarning.log , vpxa.log.

HA/FDM:

• Cannot enable HA.
• Enabling HA fails with the error:

Cannot install the vCenter agent service. Unknown installer error

Note: A misconfiguration of syslogd should not cause HA/FDM to fail in ESXi 5.0 Update 1.

Update Manager:

• Cannot scan or remediate an ESXi 5.0 host.
• Scanning or remediating an ESXi 5.0 host fails with the error:
  
  VMware vSphere Update Manager had an unknown error: Check the events and log files for detail

esxcli:

• Running the esxcli software vib list command fails with the error:

# Got no data from process /usr/lib/vmware/esxcli-software vib.list

• Running esxcli commands manually fail with the error:

# ./esxcli-software vib.list
Traceback (most recent call last):
File "./esxcli-software", line 432, in <module>
setupLogging()
File "./esxcli-software", line 78, in setupLogging
handler = logging.handlers.SysLogHandler(address='/dev/log')
File "/build/mts/release/bora-469512/bora/build/esx/release/python-2.6-lib-zip-stage/469512/visor/pylib/python2.6/logging/handlers.py", line 704, in __init__
File "/build/mts/release/bora-469512/bora/build/esx/release/python-2.6-lib-zip-stage/469512/visor/pylib/python2.6/logging/handlers.py", line 719, in _connect_unixsocket
File "<string>", line 1, in connect
socket.error: [Errno 2] No such file or directory

vmsyslogd:

• Unable to run the vmsyslogd process.
• Running the ps | grep vmsyslogd command shows no running vmsyslog daemon.
• Manually starting vmsyslog daemon fails. To restart vmsyslogd run:

# /bin/python -OO /usr/lib/vmware/vmsyslog/bin/vmsyslogd -i

The /var/log/.vmsyslogd.err log file contains messages similar to:

vmsyslog.config : ERROR ] Config error: Failed to resolve remote host: hostname.local

vmsyslog.config : ERROR ] parse config failed?

vmsyslog.main : ERROR ] Watchdog 5698 fired (child 5699 died with status 256)!

vmsyslog.main : ERROR ] Watchdog 5864 exiting

or

vmsyslog.config : ERROR ] Config error: Invalid config: could not create datstore directory
vmsyslog.config : ERROR ] parse config failed?
vmsyslog.main : ERROR ] Watchdog 5857 exiting
vmsyslog.main : ERROR ] Watchdog 5864 fired (child 5865 died with status 256)!
vmsyslog.main : ERROR ] Watchdog 5864 exiting

Environment

VMware vSphere ESXi 5.0
VMware vCenter Server 5.0.x

Cause

If the ESXi Syslog Daemon has failed or cannot start because of a misconfiguration, the host will have issues in:

• Configuring HA/FDM
• Performing esxcli commands
• Scanning or remediating the ESXi Host using Update Manager
• Normal ESXi logging

Misconfiguration of the /etc/vmsyslog.conf file results in failure of vmsyslog.

Resolution

This issue is resolved in ESXi 5.0, Patch ESXi500-201207401-BG. For more information, see VMware ESXi 5.0, Patch ESXi500-201207401-BG: Updates esx-base (2019108).

Note: After applying the patch, you may still have to restart vmsyslogd.

To work around this issue:

1. Resolve the misconfiguration in the /etc/vmsyslog.conf file. For information relating to the misconfiguration, see the /var/log/.vmsyslogd.err file.
2. Restart vmsyslog daemon using this command:
   
   /bin/python -OO /usr/lib/vmware/vmsyslog/bin/vmsyslogd -i
   
   Note: For an alternative method to restart the vmsyslogd service, see Scanning an ESXi 5.0 host fails after redirecting VMware Update Manager logs to a syslog server (2014538).

Additional Information

Determining whether the vSphere Syslog Collector service is running
Troubleshooting VMware High Availability (HA) issues in VMware vCenter Server 5.x and 6.0
Scanning an ESXi 5.x host fails after redirecting VMware Update Manager logs to a syslog server
VMware ESXi 5.0, Patch ESXi500-201207401-BG: Updates esx-base
ESXi 5.0 で syslog 構成に関する複数の問題が報告される