Total Events in Console Less Than Expected

Products

• Carbon Black App Control (formerly Cb Protection)

Issue/Introduction

The total Events shown in Reports > Events are less than expected based on the quantity setting in System Configuration > Events.

Environment

• App Control Server: All Supported Versions
• App Control Agent: All Supported Versions
• SQL Server: All Supported Versions

Cause

Typically this happens when Agents are generating more Events than expected in the given timeframe.

Resolution

Understanding Event Log Management:

It is important to understand several key factors:

1. The settings in System Configuration > Events > Event Log Management are an either/or and Events will be trimmed when either condition is met.
2. Two different timestamps are associated with each Event:
   • Timestamp: When the Event was generated on the Agent (stored in the database as date).
   • Date Received: When the Event was received by the Server (stored in the database as date_created).
3. Increasing the Event Quantity or Event Age thresholds will increase database size and potentially cause queries to take longer to complete, decreasing Console performance.
4. Patch cycles and new software deployments could generate an influx of Events by the Agents.
5. Events are deleted based on date_created rather than date. This means it would be possible to see an Event from one machine that is much older than every other Event being reported.
6. The only way to increase the Event Age without impacting database size/performance is to reduce the number of Events being generated by the Agents.

Reducing Events Generated By Agents:

1. Use SQL Server Management Studio to establish a baseline of the current Total Events and the Oldest Event:

   use das;
   select count(*) AS 'Count' from dbo.events (nolock);
   select min(date_created) 'Oldest' from dbo.events (nolock);

2. Consider discarding information about Locally Approved support files signed by Microsoft at the Agent to reduce Events generated during updates.
3. Consider blocking and suppressing PowerShell scriptpolicytest File & Event data.
4. Consider creating an ABExclusion for .NET activity.
5. Audit for Custom Rules/Rapid Configs that are potentially triggering or generating Events more frequently than necessary.
6. Use the built-in Reports to determine the Chattiest Events, Installers and Rules: https://ServerAddress/support.php > Reports > Select report & time range.
7. Consider exporting the Events to a SYSLOG Server to increase Event Retention further.

Validating Progress:

Allow a few days to pass after making changes, then execute the SQL Query from above to determine current Total Events and Oldest Event.

• If the Total Events are the same, but the Oldest Event has increased then progress is being made.
• If the Total Events are the same, and the Oldest Event is still roughly the same age, then more aggressive changes will be required.

Example:

1. Initial baseline results from the SQL Query:
   • Count: 10,032,543
   • Oldest: 5 days old
2. A few days after making changes:
   • Count: 10,024,729
   • Oldest: 7 days old

The increase in the age of the Oldest Event is a sign that progress is being made, however more aggressive changes may be desired.

•  

Additional Information

• Reminder: Once Events are trimmed from the database, they are gone.
• Events are trimmed during the Daily Prune Task each night.
• The SQL Query should be executed after patch cycles and new software deployments to help determine how these activities also impact Event Retention.
• By default the Event Log Management settings are:
  • Delete Events Older Than: 4 weeks
  • Delete If More Than: 10000000 Events
  • On Limit, Delete Oldest: 10% of Events
• The section: Events Growth, in the SQL Server Configuration Guide does provide some guidance on the estimated database file size relative to the number of Events.