Managing Zero Prevalence Pruning

Products

• Carbon Black App Control

Issue/Introduction

How to enable Pruning files that have 0 Prevalence from the database/File Catalog.

These are files that were previously seen at least once, but currently meet the following conditions:

• No copies of the file currently exist in the environment.
• There are no Approvals or Bans created for the file.
• The file is not part of any File Instance Groups or File Groups.
• The file is not part of any Snapshots.

By default these files are not removed from the database/File Catalog and the information is retained indefinitely.

Environment

• App Control Server: All Supported Versions

Resolution

Step 1: Determine Retention Period

Determine the desired retention period for file data once a file has reached 0 prevalence (has been removed from the environment). Consult your security team to assess the impact of losing file hashes and their associated history.

Step 2: Finding Total Files Eligible for Prevalence Pruning:

Standard Scoping Script

Use the attached ZeroPrevalencePruneScope.sql script to calculate the number of files eligible for Zero Prevalence pruning in your environment. This script is strictly informational and will not remove or modify any data.

1. Open SQL Server Management Studio and run it as the Carbon Black Service Account.

2. Paste the contents of ZeroPrevalencePruneScope.sql into a new query window.

3. Edit line 15 to set your desired retention period.

   • Example: SET @maxAgeDays = 90;

4. Execute the query and note the result for Total Files Eligible for Pruning. This represents the total number of files that meet the pruning criteria.

   • If you run the scope again the next day to track progress, remember to increment @maxAgeDays accordingly (e.g., SET @maxAgeDays = 91;).

Verbose Scoping Script

If the standard script reports 0 files are eligible for pruning, or if you want to understand exactly why specific files are not eligible, use the ZeroPrevalencePruneScope_verbose.sql script. This script is strictly informational and will not remove or modify any data.

1. Open a new query window and paste the contents of the attached ZeroPrevalencePruneScope_verbose.sql script.

2. Edit line 18 to set your desired retention period.

   • Example: SET @maxAgeDays = 90;

3. Export or save the results of this verbose script and provide them to Technical Support. The support team will use this output to help analyze the pruning blockers.

Step 3: Configuring Zero Prevalence Pruning:

1. Confirm there is a known-good, full backup of the App Control database.
2. Log in to the Console & navigate to https://YourServer/shepherd_config.php
3. In the drop down, find the Property: PurgeAntibodiesPeriodDays
4. Set Property Value to the amount of days determined in Step 1 (example: 90).
5. The DailyPruneTask will execute at night and will prune files that meet all conditions for Zero Prevalence within the PurgeAntibodiesPeriodDays specified.

(Optional) Step 4: Configuring Global Approval Removal:

In additon to pruning zero prevalence files, you can remove Global Approvals for any files that have Zero Prevalence and are within the PurgeAntibodiesPeriodDays specified. This feature requires Server 8.11.0 (or higher).

1. Log in to the Console & navigate to https://YourServer/shepherd_config.php
2. In the drop down, find the Property: EnableGlobalApprovalRemoval
3. Set the Property Value to 1

Additional Information

• If the DailyPruneTask is unable to complete all Zero Prevalence tasks during the allotted time (6 hours by default), it will automatically pause and resume the following night.
• The Shepherd Config Property, PurgeAntibodiesThresholdMin can be used to limit the time spent on Zero Prevalence Pruning. This may be necessary in larger environments to allow pruning tasks associated with DailyPruneTasks to execute.