rsyslog stops logging to vmdird and messages in vCenter Server 7.0

Products

• VMware vCenter Server 7.0

Issue/Introduction

Symptoms:

The syslog daemon rsyslogd is not writing to the logs it manages, such as

/var/log/vmware/messages
/var/log/vmware/vmdird/vmdird-syslog.log etc.

Cause

This issue is caused by unforeseen changes in the opensource rsyslog library employed by the vCenter appliance in 7.0+

Resolution

This issue is resolved in vCenter Server 7.0 U3o, version 7.0.3.01700, build 22357613.

Workaround:

Scripted Workaround

1. Copy the syslogConfigFix.sh script attached to this article to the problem vCenter appliance.
2. Make the script executable by running:

   # chmod +x syslogConfigFix.sh

3. Execute the script by running:

   # ./syslogConfigFix.sh

For Example:

root@vcsa_name [ ~ ]# chmod +x syslogConfigFix.sh
root@vcsa_name [ ~ ]# ./syslogConfigFix.sh

Manual Workaround

1. Take a backup of /etc/rsyslog.conf

   # cp /etc/rsyslog.conf /etc/rsyslog.conf.bak
   

2. In /etc/rsyslog.conf, check for *.* ?esxLoc;esxFmt and replace it with if ($hostname != $$myhostname ) then ?esxLoc;esxFmt
3. Save the file
4. Take a backup of /etc/vmware-syslog/syslog.conf 

   # cp /etc/vmware-syslog/syslog.conf /etc/vmware-syslog/syslog.conf.bak

5. In /etc/vmware-syslog/syslog.conf, check for & stop at the bottom and remove it.
6. Restart the rsyslog service:

   # systemctl restart rsyslog

After performing the above workaround, check /var/log/vmware/vmdird/vmdird-syslog.log for logging. If there is still no logging, vmdird service needs to be restarted.

service-control --restart vmdird

Additional Information

Impact/Risks:

Remote logging will not be collected for vmdird service(s).