Configure Data Collector disk space allocation for polled data cache during DA outage

Products

• CA Performance Management
• Network Observability

Issue/Introduction

The Data Collector will cache up to 50% of configured Data Collector memory by default per per How Data Collector caches data when the Data Aggregator is down

This can be increased beyond this as needed.

Environment

DX NetOps Performance Management all versions

Resolution

To increase the configured value follow these steps.

1. On the DC create a file named com.ca.im.core.jms.health.cfg in the (default path) directory:
   
   /opt/IMDataCollector/apache-karaf/etc (for 25.4.3 or earlier)
   
   /opt/IMDataCollector/config (for 25.4.4 or later)
   
   
   • This file is not created by default.
   • This file may be removed during an upgrade.
     
     
2. Add the following line in the file. Specify the amount of  space you would like to use for caching by adding an entry for customMaxDiskSpace.
   • New line:
     • customMaxDiskSpace=<SPACE><UNIT>


   • Example configured for 10 GB allocation.
     • customMaxDiskSpace=10G



3. Save the file changes.

The file creation and edits are tracked in (for 25.4.3 and earlier) the DC karaf.log file file found in:

/opt/IMDataCollector/apache-karaf/data/log.

For 25.4.4 and later, we have moved from Apache-Karaf to Springboot, so the location is in:

/opt/IMDataCollector/data/logs/netops-data-aggregator.log

 

You can create the file one of two ways and tracking log messages to see it get read into the running process.

1. While in the correct directory (as per above), open/create the file with "vi com.ca.im.core.jms.health.cfg". Enter edit mode, enter the config line, and save the changes.
2. Create the file in a different directory. When complete, copy it into the
   
   /opt/IMDataCollector/apache-karaf/etc directory for 25.4.3 or earlier,  
   
   or /opt/IMDataCollector/config for 25.4.4 or later.

In the karaf.log / netops-data-aggregator.log, we'll see messages similar to the following when the file is added or edited.

025-03-10T16:39:30,866 | INFO  | -karaf-4.4.6/etc | fileinstall                      | install.internal.Util$OsgiLogger  205 | 12 - org.apache.felix.fileinstall - 3.7.4 |  | Creating configuration {com.ca.im.core.jms.health} from /opt/IMDataCollector/apache-karaf-4.4.6/etc/com.ca.im.core.jms.health.cfg

2025-03-10T16:39:30,871 | INFO  | core.jms.health) | JmsBrokerHealthService           | ms.health.JmsBrokerHealthService   57 | 31 - com.ca.im.common.core.jms - 24.3.5.RELEASE-6 |  | JMS Health: set PRQ maxdiskspace =10G

And when the file is deleted, the message should be similar to:

2025-03-10T16:40:12,873 | INFO  | -karaf-4.4.6/etc | fileinstall                      | install.internal.Util$OsgiLogger  205 | 12 - org.apache.felix.fileinstall - 3.7.4 |  | Deleting configuration {com.ca.im.core.jms.health} from /opt/IMDataCollector/apache-karaf-4.4.6/etc/com.ca.im.core.jms.health.cfg

Additional Information

KB Article: How to confirm that the DC is collecting data

This configuration is not tested.

By extending the cache, it will take longer to reach live data being seen, as it must process the cached data first.

Thresholding times may go up as more timestamps would need to be processed in a shorter amount of time.

This may tax the thresholding engine, and possibly leading it to move to a state of degraded and ultimately disabled.