On-Demand-Log file management and disk space usage in Data Collector

Products

• CA Performance Management
• Network Observability

Issue/Introduction

Data Collector hosts generate binary log files for discovery debug data that persist through service restarts․․․․․․․ These logs are stored in the On-Demand-Log directory and can consume significant disk space if not managed properly.

SYMPTOMS:

• High disk space consumption in the [datacollectorInstallDir]/dcdebug/On-Demand-Log directory․

• Multiple files named by device IP address present in the log path․

• Large .gz files associated with discovery messaging․

CONTEXT: This occurs on Data Collector hosts during device discovery processes.

IMPACT: Excessive log growth can lead to disk space exhaustion on the Data Collector server.

Environment

All supported DX NetOps Performance Management releases

Cause

The On-Demand-Log directory stores persistent discovery messaging for individual devices.․․․․․․․ Large uncompressed data volumes within .gz files (up to 40 MB uncompressed) can lead to higher than expected disk utilization.

Resolution

PREREQUISITES:

• Root or sudo access to the Data Collector host․

• Knowledge of the Data Collector installation directory․

LOG FILE BEHAVIOR:

1. Each device discovery creates a .log file limited to 2 MB.

2. Once the .log file reaches 2 MB, it is compressed into a .gz file.

3. The .gz file maintains up to 40 MB of uncompressed data; adding new data triggers the removal of the oldest 2 MB to maintain this limit.

4. Total persistent storage per discovered IP is approximately 4 MB (2 MB .log + 2 MB .gz).

STEPS:

1. MANAGE EXISTING LOG FILES:

   • You can safely delete files in the On-Demand-Log directory while the Data Collector is running.

   • NOTE: Deleting these files results in the loss of historical discovery messaging used for troubleshooting.

   • Files are automatically removed when the managed device is deleted from the system.

2. MODIFY LOG SIZE LIMITS: Path: [datacollectorInstallDir]/IMDataCollector/config/com.ca.im.dm.core.collector.cfg

   • Open the configuration file and add or modify the following parameter: collector-memory-log-size-on-demand=[size-in-KB]

   • Example: To set the limit to 1 MB, use 1024; for 512 KB, use 512.

VERIFY SUCCESS:

• Monitor the On-Demand-Log directory to ensure new files adhere to the adjusted size limits.

Additional Information

Upgrading an already installed DC but the reported disk size requirements per the installation seem excessive, or higher than expected? Please see Knowledge Base Article Unexpected amount of space required for DC upgrade if disk consumption issues related to these logs is a concern.