Troubleshooting Agent Performance Issues

Products

• Carbon Black App Control (formerly Cb Protection)

Issue/Introduction

Step-by-step guidance on troubleshooting Agent performance issues.

Environment

• App Control Console: All Versions
• App Control Agent: All Supported Versions
• Microsoft Windows: All Supported Versions
• Apple MacOS: All Supported Versions
• Linux: All Supported Versions

Resolution

Initial Steps

1. Verify the endpoint
   • is using a supported and compatible Agent version.
   • is fully Initialized, as the process itself can be resource intensive.
   • has Agent Exclusions present in any installed third-party security applications (e.g. antivirus, firewall, real-time scanner, vulnerability scanner, etc.).
2. Verify the issue can be recreated with the Agent service fully stopped and the driver unloaded
   • If the issue persists, resolve the underlying issue first before proceeding.
   • If the issue does not persist, proceed with troubleshooting.

Recent Issues

1. Windows
   • Versions 8.9.0-8.10.0: Upgrade to 8.10.2+ or implement the two Agent Configs as outlined here.
   • Review guidance for large, non-executable files (vmdk, vhd, bak, vib, vbk, etc) 
   • Avoid use of Authenticated User in Custom Rules, see: Custom Rules That Should Be Avoided
2. Linux
   • Version 8.8.4+: Review performance considerations due to architecture change in 8.8.x
   • Versions 8.8.0-8.8.2: Upgrade to 8.8.4+
   • Verify the Agent's db-wal growth is not negatively impacting performance.
   • Verify the Linux System Performance Rapid Config is properly configured and applied.

General Guidance/Continued Investigation

1. Application specific considerations:
   • Microsoft Exchange Server
   • Microsoft SQL Server
   • Visual Studio
   • Windows Updates
   • Performance Hits and Slow Login on Citrix Devices
2. Verify the Agent's CL Version is up-to-date to ensure any/all changes are applied to the Agent.
3. Check for Custom Rules that should be avoided due to negative impacts on Agent performance, examples:
   • Using Authenticated User in Custom Rules will result in unnecessary Rule Expansion
   • Performance Optimization Rules should avoid including Interesting Files to prevent stalls on executions.
4. Upgrade to the latest Agent version to eliminate performance enhancements and other Resolved Issues.
   • Implementing workarounds or exclusions for Resolved Issues may reduce the security posture of the Agent.
5. For Windows, use Procmon to identify processes that could be included in a Performance Optimization Rule.
   • A Performance Optimization Rule will ignore Modifying Operations (Write, Write Delayed, Delete, Rename, Create New, Mmap Write) on the Path or File by the specified Process.

 

If the issue persists, collect & provide the following information in a Support case:

• Steps taken/findings from the list above
• All Background Information and Logs from: Agent Performance Logs

Additional Information

• Agent Performance Logs should be collected from the latest available Agent version while still able to replicate the issue.