Rsyslog stops streaming logs to remote syslog server

Products

• VMware Usage Meter

Issue/Introduction

To be aware of the known issue.

Symptoms:

• Only logs from CAP-related services are sent to the remote syslog server.
• System logs stop streaming to the remote log server after some time.
• The issue is reproducible on both new deployment and upgraded deployment.

Steps to reproduce:

Deploy an appliance based on CAP 4.0(say Usage Meter 4.8) and configure the Syslog server via CAP VAMI UI.
Observe syslog server logs.

Environment

VMware vCloud Usage Meter 4.x

Cause

With the latest rsyslog in CAP 4.0, the rsyslog service configuration and the remote server configuration file need to be aligned, so that it streams all the logs to the remote server.

Resolution

This issue is fixed in Usage Meter 9.0 release - Release Notes Link

Workaround:
Greenfield Deployment :

Once the remote syslog is configured on a new deployed appliance, we need to remove the `& stop` in the last line of
/etc/vmware/cap/cap_am/rsyslog/remote-server.conf post remote server configuration.  

In the  /etc/rsyslog.conf move line $IncludeConfig /etc/vmware/cap/cap_am/rsyslog/remote-server.conf after `$DefaultNetstreamDriverKeyFile  /etc/vmware/cap/cap_am/server.pem`

Post making above changes, restart rsyslog server :
systemctl restart rsyslog

Brownfield Deployment :

If the upgraded appliance has remote syslog configured, remove the `& stop` in the last line of
/etc/vmware/cap/cap_am/rsyslog/remote-server.conf post remote server configuration.  

In /etc/rsyslog.conf move line $IncludeConfig /etc/vmware/cap/cap_am/rsyslog/remote-server.conf after `$DefaultNetstreamDriverKeyFile  /etc/vmware/cap/cap_am/server.pem`

Post making above changes, restart rsyslog server :
systemctl restart rsyslog

Additional Information

N/A

Impact/Risks:
Certain logs are not getting forwarded to the remote server