How is Data Retention Determined?

Products

• Carbon Black EDR (formerly Cb Response)

Issue/Introduction

How is Data Retention Determined for EDR servers?

Environment

• EDR Server:  All Supported Versions 

Resolution

Event retention is based on the first hit configuration in /etc/cb/cb.conf. 

• MaxEventStoreDays: Default is set to 30 days. If event data is older than 30 days the oldest event core will be removed or converted to cold storage if enabled.
• MaxEventStoreSizeInMB: Similar to MaxEventStoreSizeInPercent, if the size of the total event store hits this parameter, the oldest event cores will be removed or converted to cold storage until the size is back under the limit.
• MaxEventStoreSizeInPercent: Default is 90%. Determines if the partitioned disk size where the event cores reside has hit the total disk percentage. This is to avoid filling the disk, causing the EDR  Server to no longer function. The oldest core will be deleted or converted to cold storage if enabled.
• MinAvailableSizeInMB: This is set when AlwaysDeleteColdPartitions is set to False. This parameter when set will remove the oldest warm core instead of converting to cold storage if conditions are met.
• SolrTimePartitioningActivePartitions: This setting controls the number of actively searched Solr partitions that remain in the query index i.e warm cores. 
• SolrTimePartitioningMinutes: This setting controls how many minutes a partition will be written to before moving to the next writer core.

Additional Information

• The server checks every 15 minutes to determine if data needs to be purged. 
• By default event cores consist of 3 days of data with a total of 10 cores when 30 days of retention is set. When purging, the oldest 3 days of data will be removed.
• For cold core storage: How to enable event cold storage
• If you have the cb-event-forwarder installed, follow this article to avoid hitting the disk size configs early: Disk Full Due to JSON Files
• Physical Binaries stored in /var/cb/data/modulestore can take up disk space causing the disk size configs to purge cores early: How To Purge Modulestore Binary Files
• See other server configuration settings in EDR Server Configuration Guide