How to calculate Retention Days in Aria Operations for Logs

Products

• VMware Aria Suite

Issue/Introduction

This article highlights steps on how you can calculate the retention days in Aria Operations for logs as per your configuration

Resolution

To calculate the retention period, gather the below information:

1. Number of nodes in the cluster
2. Number of Events Ingestion Rate (per second) from the UI under Management -> System Monitor - Statistics 
3. Output of 'df -h' from all the nodes in the cluster

Retention days in an Aria Logs cluster is equal to Total Space / Daily Consumption where Total Space is the accumulative storage dedicated to /storage/core on all nodes. 

To simplify this, here's an example:

1. Number of nodes in the cluster = 3
2. Number of Events Ingestion rate (per second) = 18,000
3. Total Space on /storage/core on one node ~ 3.4 TB. 
4. Total Space on the cluster = Total Space on a node * Total nodes in the cluster = 3.4 * 3 ~ 10 TB

Daily space consumption is 86400 * 18000 * 400 ~ 0.65 TB

where, 86400 are the total second in a day
18000 is the number of events ingested per second
and 400 is the average event/log size

So in this case, Retention Days = Total Space / Daily Consumption = 10 / 0.65 ~ 15

Using the same formula, we can calculate the total space needed on the cluster to increase retention days.

Just keep in mind that we have a 4TB storage limit/recommendation per node and once a hard disk is added to a node in Aria Logs cluster, it is not supported to remove it.