Check Point sk36846 - Interpreting high softIRQ values

book

Article ID: 168104

calendar_today

Updated On:

Products

XOS

Issue/Introduction

This article presents information from Check Point on high softIRQ CPU usage on systems running a Check Point security application.This article presents information from Check Point on high softIRQ CPU usage on systems running a Check Point security application.

Cause

Goal: To understand high softIRQ CPU usage on a system running a Check Point security application.

softIRQ CPU usage is a direct result of kernel interrupt activity.

If you observe high softIRQ CPU usage on a system running a Check Point security application (ie. VPN-1 NGX, VSX NGX, etc.), Crossbeam highly recommends consulting the following SK article published by Check Point support.

Resolution

The Check Point SK article "sk36846" articulates general causes of high softIRQ. In all cases, you should consider the impact of any of the recommendations to remedy the condition before taking appropriate action. Contact Crossbeam or Check Point Technical Support for more information.
---------------------------------------------------------------------

sk36846
(from Check Point Usercenter) -

softIRQ is a value that indicates the level of kernel activity. Because the stateful inspection and packet handling is made on the kernel level, high softIRQ values result if one of the following actions is performed on a high amount of traffic:

1. Deep protocol inspection.

2. NAT.

3. VPN encryption/decryption.

4. Synchronization (among cluster members).

5. Log generation.



When consistently high softIRQ values are seen under the top command, check the following things:

1. The high softIRQ values correlate with the amount of traffic passing through the gateway at that time.
Meaning:
Large amount of traffic = high softIRQ
Small amount of traffic = low softIRQ

2. The softIRQ values are constantly higher than 90% during peak times.

3. RX drops seen under 'ifconfig' or 'netstat -i' commands.

4. Traffic latency is occasionally experienced.



Note: Ocassional softIRQ levels lower than 90% during peak traffic load do not necessarily indicate a problem. This probably reflects normal FW-1 activity. If one or more symptoms is seen, modify the configuration in order to decrease the level of kernel activity.

The following steps should be taken, one at a time and not necessarily in order, until the softIRQ values decrease to an acceptable level:

1. When using a machine with multiple cores, enable SecureXL, if possible, and optimize it so that as much traffic as possible will be accelerated and templates will be created (refer to sk32578).

2. Reorder the rule base so that frequently used rules will be at the top of the list.

3. Disable the handler (protocol type) on frequently used services.

4. Disable SmartDefense, Web Intelligence and\or Content Inspection.

5. In cluster environments, disable synchronization of services that are not require synchronization, for example HTTP and DNS-UDP.

6. Disable logging for less important rules.

7. In case of RX drops on intel pro/1000 family NICs refer to sk25921.

8. Enable CoreXL if possible.

Workaround

N/A