High CPU on all enrolled SEP clients with more than 500 SHA256 Deny/Allow List entries in EDR

Products

• Endpoint Protection with Endpoint Detection and Response
• Endpoint Detection and Response
• Endpoint Protection
• Advanced Threat Protection Platform

Issue/Introduction

- When creating a SHA-256 Deny/Allow List entry within Endpoint Detection and Response (EDR), the entry appears in the Exceptions policies within Symantec Endpoint Manager (SEPM).
- When over approximately 500 entries are present in the Exceptions policies within SEPM, high CPU symptoms impact all enrolled SEP clients.

Cause

• In EDR 4.5, EDR added the capability to locally enforce deny list at SEP endpoints for SHA256 hashes via SEP Exception policies.
• The expected use case for Deny/Allow List feature is to target files identified within the customer environment by EDR Search or Incidents.
• At design, the number of SHA256 entries was not expected to exceed to SEP endpoint capacity of approximately 500 entries.
• The Deny/Allow List limit within EDR UI of 65,000 entries is originally based on accepting up to 65,000 MD5 hashes.

Resolution

 

To resolve this issue, do one of the following:

• On version 4.6.7 and later, login to the Command Line Interface (CLI) as 'admin' and type:
  exception_list disable
  
  
• On version 4.5.0, install patch 4.6.5-1. EDR must be on version 4.6.5 when applying this patch.

• • If you have a high number of clients and a low number of Exception policies, perform Patch Method #1
    NOTE: This method avoids the impact of un-enrolling and re-enrolling all SEP endpoints with EDR
    
    
  • If you have a low number of clients or a high number of Exception policies, perform Patch Method #2
    NOTE: This method avoids the need to manually edit exception policies within SEPM or manually edit the Deny List within EDR

 

Patch Method 1:

1. (OPTIONAL) Within EDR UI, export the current Deny List
2. Install patch 4.6.5-1 at the command line interface (CLI) of EDR
3. In the SEPM Exceptions policy, click the Source column to sort exceptions by Source
4. Click the top entry with a source that starts with "EDR"
5. Scroll down to locate the bottom entry with a source that starts with "EDR"
6. While holding the <SHIFT> key, click the bottom entry with a source that starts with "EDR"
7. Below the list of exceptions, click the Delete button
8. When the Delete Exception dialog box appears, click the Delete button on the Delete Exception dialog box.
9. Repeat steps 3-8 for each Exception policy within SEPM
10. Wait 24 hours and monitor to ensure high CPU stops and does not return

 

Patch Method 2:

1. (OPTIONAL) Within EDR UI, export the current Deny List
2. Remove the SEPM Controller connection
3. Within SEPM, monitor Exception policies to confirm when the last of the SHA-256 entries from EDR are removed.
4. Install patch 4.6.5-1 at the command line interface (CLI) of EDR
5. Re-add the SEPM Controller connection, with a pilot group of endpoints
6. Wait 24 hours and monitor the pilot group to ensure high CPU does not return
7. Re-add the remaining endpoint groups

 

What does the patch do?
- Disable sending sha256 hashes to the Exception policies of SEPM. 
  As EDR will no longer send SHA256 hashes to Exception policies within SEPM, local blocking of previously blocked executable files may require entering md5 hashes for those files on the Deny List of EDR instead. EDR will not be able to block non-PE files.

 

To install patch 4.6.5-1

1. At the admin CLI of EDR, type:
   show -v
   
   
2. If version is less than 4.6.5, then type:
   update download
   
   
3. If no errors occur during update download, type:
   update install
   
   
4. Updating the software version may require up to two reboots of EDR appliance before continuing.
5. To confirm the installed patches, type:
   patch list_installed
   
   
6. If "atp-patch-4.6.5-1" appears in the output, the EDR appliance is already patched for this issue. No further action is needed for this particular EDR appliance.
7. To check for the patch in the download repository, type:
   patch list
   
   
8. If "atp-patch-4.6.5-1" does not appear in the download repository, please contact support for further assistance and reference KB #. Also copy and paste the output from this command into the case comments.
9. To download the patch, type:
   patch download atp-patch-4.6.5-1
   
   
10. If the last three lines from patch download are not as follows, create a support case and paste the output into the case comments.
       atp-patch-4.6.5-1.x86_64.rpm                               | 718 kB   00:01 ETA
       Download succeeded
       Function: main returned success
    
    
11. To install the patch, type:
    patch install atp-patch-4.6.5-1
    
    
12. If patch install does not include the following two lines, create a new support case and copy and paste the output from the patch install command into the comments.
       Patch installation Success!
       Function: do_install returned success