Observe impacts on ingestion, permanent disk blocks which never disappears

Products

• VMware Aria Suite

Issue/Introduction

Observe impacts on log ingestion, permanent disk blocks which never disappears.

Environment

Aria Operations for Logs 8.18

Cause

1. Having configured many log forwarders with plenty of filters may sometimes affect ingestion.
2. The level of complexity of the filters causes the issue.

Resolution

The resolution will be changing/increasing the ingestion rate and/or enlarge the cluster.

 

An example that is tested:
The cluster has 3 large (16 vCPUs and 32 G Memory) nodes and each node has an ingestion rate of about 18k events per second:

• In case fields which utilize extracted fields, such as " text matches regex *process*" need to be used in log forwarders: recommend having up to 5 forwarders with 8 fields which use extracted fields.
• In case filters which utilize static fields, such as " hostname starts with vc* " need to be used in log forwarders: recommend having up to 15 forwarders with 8 filters which use static fields.