Introscope SmartStor database metrics volume per day

Products

• CA Application Performance Management (APM / Wily / Introscope)

Issue/Introduction

You want to know the size of metrics volume that APM environment is receiving per day.

It is both the number of metrics and total metrics volume in MB or GB per day the APM environment is receiving for APM cluster.

Environment

APM 10.8 SP1  cluster

Resolution

Information related to the APM smartstor database and how much data it is receiving from monitored agents etc.

To get information on the number of metrics and total metrics volume in MB or GB per day the APM environment is receiving in your APM cluster, please review the following.

Metric paths:

*SuperDomain* > Custom Metric Host (... > Custom Metric Proces... > Custom Metric Agent ... > Enterprise Manager > Data Store > SmartStor > Metadata > Agent Metric Cache Size

*SuperDomain* > Custom Metric Host (... > Custom Metric Proces... > Custom Metric Agent ... > Enterprise Manager > Data Store > SmartStor > Metadata > Cached Metric Count

*SuperDomain* > Custom Metric Host (... > Custom Metric Proces... > Custom Metric Agent ... > Enterprise Manager > Data Store > SmartStor > SmartStor Disk Usage (mb)

*SuperDomain* > Custom Metric Host (... > Custom Metric Proces... > Custom Metric Agent ... > Enterprise Manager > Connections > Number of Metrics

https://techdocs.broadcom.com/us/en/ca-enterprise-software/it-operations-management/application-performance-management/10-8/ca-apm-sizing-and-performance/enterprise-manager-cluster-sizing-and-performance-considerations.html#concept.dita_05cb26fd65a33ac613e840f3884edb80ac611036_SmartStorandEnterpriseManagerPerformance
SmartStor and Enterprise Manager Performance
SmartStor Spooling
SmartStor Reperiodization

You can review the actual SmartStor database directory to get better idea about the total metrics volume in MB or GB per day the APM environment is receiving.

Please review: <APM-home>/data directory.

The important files are *.data and *.spool files. You need to check these files on each EM (both collector and MOM).

You can run the following script to monitor the size of *.data and *.spool files on each EM (both collector and MOM).

Run the command within <APM-home>/data directory. You many need to run the command for 24 hours to get better idea.

while true; do printf "\n=====================\n" >> /tmp/output.log; date >> /tmp/output.log; ls -l >> /tmp/output.log; sleep 30; done &

Please note down the PID of the command so that once it completes collecting the data you can kill it.

Here is an example from a collector.

-rw-r--r--. 1 root root   162573 Sep 18 10:00 1758203085000.data
-rw-r--r--. 1 root root   674923 Sep 18 11:00 1758204015000.data
-rw-r--r--. 1 root root   687626 Sep 18 12:00 1758207615000.data
-rw-r--r--. 1 root root   687418 Sep 18 13:00 1758211215000.data
-rw-r--r--. 1 root root   686397 Sep 18 14:00 1758214815000.data
-rw-r--r--. 1 root root   686481 Sep 18 15:00 1758218415000.data
-rw-r--r--. 1 root root  4523533 Sep 18 16:00 1758222015000.data
-rw-r--r--. 1 root root  6623227 Sep 18 17:00 1758225615000.data
-rw-r--r--. 1 root root 78901692 Sep 18 17:40 1758229215000.spool
drwxr-xr-x. 2 root root     4096 Sep 18 10:14 archive
drwxr-xr-x. 2 root root     8192 Sep 18 09:44 metadata
-rw-r--r--. 1 root root  1642496 Sep 18 17:40 variance.db

Each hour incoming data is being saved in the 1758229215000.spool, and at the start of the each hour data is moved to 1758225615000.data.
EM reformats the .spool file into a SmartStor .data file at the start of the each hour.