We'd like to know how to confirm that the DC is still collecting data even when the DC is not able to connect to the other nodes.
Release : 22.2.x and 23.3.x
The Data Collector will collect data if it can't connect to the Data Aggregator as long as it's running. If the Data Collector is down and the Data Aggregator is down, the Data Aggregator will need to be up and running to start to the Data Collector.
You can also check the PollSummary.log in the Data Collector logs, this should be logging each poll cycle per Metric Family/poll group.
When the queue is full the Data Collector will start dropping the oldest ActiveMQ poll messages if the ActiveMQ queue is getting full on the allocated disk space for it. We
either use 50% of the XMX for the Data Collector or 90% of the disk free, whichever is lowest.
You can override that setting and tell the Data Collector to cache more data than 50% DC XMX.
Edit/create
/opt/IMDataCollector/apache-karaf/*/etc/com.ca.im.core.jms.health.cfgAdd:customMaxDiskSpace=<amount>GExample to use 100G if there is 100G on disk.
This allows for more data to be saved when connection is down for say a DA upgrade or long outage between DA/DC.customMaxDiskSpace=100G