There is an imbalance of syslog events in Aria Operations for Logs causing dropped events

Products

• VMware Aria Suite

Issue/Introduction

In an Aria Operations for Logs cluster, there might be a few nodes that are imbalanced and getting 5x to 10x the amount of syslog traffic as the other nodes according to the Management > System Monitor > Statistics page.

The Management >System Monitor > Statistics > Show advanced statistics pane shows that syslog events are being dropped as a result.

Environment

Aria Operations for Logs 8.18.x

Cause

This may be caused by fat sources of data: a single source of logging that is sending an overwhelming amount of syslog events.  This happens because load balancers will typically balance sessions for the syslog events by IP source/port/protocol and consider that a single session, thus sending the single source to one single node.

Resolution

Use the Explore Logs tab to get "Count of Events" by source in a non time series.

1. Select "over time," this will bring up a drop down menu.  Select Non-time series, group by source.



2. Examine the tallest bars for the source of the most logs within the given time period and determine why the source is sending an overwhelming amount of logs in comparison to other sources.