Log forwarding stops working after log rotation

Products

• VMware vCenter Server 7.0
• VMware vCenter Server 8.0

Issue/Introduction

After log rotation is triggered, the following files keep writing logs but stop forwarding logs due to a syslog issue.

• /opt/vmware/var/log/lighttpd/access.log

• /opt/vmware/var/log/lighttpd/error.log

• /var/log/audit/audit.log

• /var/log/vmware/applmgmt-audit/applmgmt-audit.log

Note: This issue may also occur with other logs.

 

 

Resolution

To resolve this issue, adding both options deleteStateOnFileDelete="on" and reopenOnTruncate="on" to /etc/vmware-syslog/vmware-services-*.conf.
/etc/vmware-syslog/vmware-services-*.conf are the configuration files where the target logs are defined.

The following is an example workaround for /opt/vmware/var/log/lighttpd/access.log.
/opt/vmware/var/log/lighttpd/access.log is defined in /etc/vmware-syslog/vmware-services-applmgmt.conf.


Workaround for /opt/vmware/var/log/lighttpd/access.log

1. Stop rsyslog and syslog.socket service
   systemctl stop syslog.socket rsyslog.service
   
   
   
2. Create config backup
   cp /etc/vmware-syslog/vmware-services-applmgmt.conf /etc/vmware-syslog/vmware-services-applmgmt.conf.bak
   
   
   
3. Add both options deleteStateOnFileDelete="on" and reopenOnTruncate="on" to /etc/vmware-syslog/vmware-services-applmgmt.conf.
   
   
   For example
   

   Before

   #vami-access.log
input(type="imfile"
File="/opt/vmware/var/log/lighttpd/access.log"
Tag="vami-access"
Severity="info"
Facility="local0")

   After

   #vami-access.log
input(type="imfile"
File="/opt/vmware/var/log/lighttpd/access.log"
Tag="vami-access"
Severity="info"
deleteStateOnFileDelete="on"
reopenOnTruncate="on"
Facility="local0")

4. Delete the imfile-state file
   
   find /var/log/vmware/rsyslogd/ -name "imfile-state*" | xargs -i rm -f {}
   
   
5. Start rsyslog and syslog.socket service
   
   rsyslog and syslog.socket service