Console Performance Issues and Growing Agent Backlog

Products

• Carbon Black App Control (formerly Cb Protection)

Issue/Introduction

• Console pages are slower than normal to load.
• Agent Backlog is growing.
• Historical ServerLog.bt9 includes "gave up" messages when inserting an Antibody into some table, example:

  (8332 ReportABList Thread 1)After 0 retries, gave up on re-running statement : SET NOCOUNT ON;{CALL dbo.InsertAntibody (?,?,?,?,?,?,?,?,?,?,?,?,?,?)}.

• High Debug ServerLog.bt9 includes messages similar to any of the following:

  (5244 ReportABList Thread 1) Statement returned error [102]: [Microsoft][SQL Server Native Client 11.0][SQL Server]Incorrect syntax near '288241'.
  Statement: INSERT INTO dbo.temp_antibody_instances VALUES

  (11824 ReportABList Thread 0) Statement returned error [102]: [Microsoft][SQL Server Native Client 11.0][SQL Server]Incorrect syntax near 'VALU'.
  Statement: INSERT INTO dbo.temp_antibody_instances VALU

  (812 ReportABList Thread 0)ABStorage::_LoadAntibodyFromDB - max size reached bufferSizeUsed = 2794, fileOpCount = 20

Environment

• App Control Server: Versions 8.9.0 - 8.10.x
• App Control Agent: Versions 8.0.x - 8.9.x

Cause

The Agent is reporting very large Yara Tags that exceed 1024 characters, causing SQL Statement truncation issues and errors during Server processing.

Resolution

A permanent fix will be included in a future release of both the Agent and Server.

• The Server side investigation was tracked under EPCB-20744 and is anticipated for inclusion in Server 8.11.0.
• The Agent side investigation was tracked under EPCB-20745 and is anticipated for inclusion in Agent 8.10.0.

In the meantime, temporarily adjust the Shepherd Config, MaxFileOpBufferMessages

1. Log in to the Console and navigate to: https://ServerAddress/shepherd_config.php
2. Select the Property: MaxFileOpBufferMessages
3. Change the Value to: 2
4. Restart the services:
   • Carbon Black App Control Server
   • Carbon Black App Control Reporter
5. Log back in to the Console test overall performance.

Additional Information

This Shepherd Config should be reverted to the default Value of 20 after upgrading to Server 8.11.0+