AXA prune ingestion logs

Products

• CA App Experience Analytics

Issue/Introduction

How do I prune the ingestion logs in AXA?  The generic logs, which is the largest, have not increased since install.  The only other logs being ingested are event logs from one server.

Environment

Release : 17.3

Component : APP EXPERIENCE ANALYTICS ENGINE

Resolution

The default retention setting for logs are 45 days. If needed,  change the setting as below. 

 

The below procedure can be used to change the retention setting.

---------------------

go to Deployments-->estutils pod

ao_aum_discovered_data_1.1=2G,ao_itoa_metrics_uim_1=5,ao_itoa_metrics_anomaly_1=5,ao_itoa_alarms_all_1=5,ao_itoa_metrics_agg_level1_1=5,ao_itoa_metrics_custom_1=5,ao_itoa_metrics_capm_reachability_4=5,ao_itoa_alarms_anomaly_1=5

you can give something like above

ao_itoa_logs_generic_1=5

check for

INDEX_LIMIT

variable

DEFAULT_RETENTION_PERIOD

you can change this also

----------------------

 

This case was related to Log Analytics which is shipped by AXA installer.

• Coming to the question of log purge, Elasticsearch index will rollover, by default, at 30 GB. In this situation, your Generic log index is of 8 GB. We still have almost 22 GB to start log purge.
• Any logs which are ingested and have no patterns associated, will go on to generic index. Only the indices which are rolled over will be applicable to purging. Thus,  the present Generic logs are yet to roll over.
• Coming to the Deployments ->estutils pod, this is OSE display and need to be done through commend prompt or terminal. But customers environment it 17.3.2, this version doesn't have OSE display support as it is a older version. 
• Thus, we can say that until index are rolled over we can't do purging. And this Generic Indices will be filled only when logs which are ingested and have no patterns associated , will go on to generic index. 
• To note, 30GB is for all logs ex, generic, sys, event logs.  Once the rollover happens, purging will start as per date and data stored.