Performance analysis in VPN-1 Power/UTM NGX R65 - Check Point sk33781

book

Article ID: 167944

calendar_today

Updated On:

Products

XOS

Issue/Introduction

Describes the steps to understand and mitigate high CPU usage in VPN-1 Power/UTM NGX R65 (Check Point sk33781) with references to other Crossbeam articles.High CPU usage on a VAP running Check Point software.

Cause

A VAP exhibits high CPU usage.

Resolution

High CPU usage is observed on a VAP running Check Point software compared to the expected CPU usage. Refer to the mitigation steps outlined in the Check Point article sk33781 prior contacting the Crossbeam Systems Support team.

Refer to the Check Point article sk33781 on the Check Point Support site for additional information.

For convenience, a version of Check Point sk33781 has been pasted below:
#####
Solution ID: sk33781
Performance analysis in VPN-1 Power/UTM NGX R65 and Security Gateway R70
Last Modified: 22-wrz-2009

Solution

The following is a list of commands that can be run to monitor the Performance of the Security Gateway.

top - Verify the CPU/memory usage.

Note: CPUs above 90% may be problematic and may be the result of:

* Sim affinity that is not set optimally.

* F2F traffic.

Identify the top CPU consuming processes from the output of this command. Verify the memory consumption.

ethtool -S ethx - Checks the NICs statistics.

Verify the rx/tx drop/error statistics.

ethtool -i ethx - Checks the NICs driver version.

Verify that the version is "NAPI" (optimally for performance).

cat /proc/interrupts

Verify that the NICs do not share the same IRQ number, which is problematic with affinity.

ethtool ethx - Checks speed/full-duplex setting.

cat /proc/cpuinfo - CPU information.

Verify that HyperThreading is disabled when working with Performance Pack.

/var/log/messages - system log file

Verify the NICs model. The model can be found in this file after reboot. It is recommended to use the PCI-Express NICs (called PCIe, or PCI-E), because they have better performance than PCI eXtended NICs (called PCI-X).

SecureXL Templates:

* Verify that templates are not disabled (use the fwaccel stat command for this purpose).

* For further information regarding Templates mechanism, refer to sk32578 - SecureXL Mechanism.

Delayed notification:


* In the ClusterXL configuration, the Delayed Notification feature is disabled by default. Enabling this feature improves performance (at the cost of connections' redundancy, which can be tuned using delayed notifications expiration timeout).

* The fwaccel stats command indicates the number of delayed connections.

* The fwaccel templates command indicates the delayed time for each template under the DLY entry.


Non-accelerated traffic analysis:


* Use the fwaccel stats command to verify the amount of non-accelerated traffic compared to accelerated traffic.

* Use the sim dbg + f2f command to understand the possible reasons for the non-accelerated traffic.


Sim affinity manual settings:


Refer to the Sim affinity section in the Performance Pack User Guide.

Improving performance with ClusterXL LS Unicast (Changing the load in the pivot):


Using GuiDBedit, change the value of the Pivot_overhead attribute on the cluster object.
The default value of this attribute is 20, which means that by default 20% of the Pivot member work is done for Pivot operations (such as forwarding some of the traffic to the Non-Pivot members). The remaining 80% of the Pivot member work capacity is used for handling traffic by itself.
To release the Pivot for more Pivot operations, increase the Pivot_overhead attribute value.

Examples:

1. If the Pivot_overhead is 0%, and there are two members, each member will handle 50% of the traffic.

2. If the Pivot_overhead is 20%, and there are two members, the Pivot member will handle 30% of the traffic, and the non-pivot will handle 70% of the traffic.



SmartDefense configuration on performance:


Refer to the SmartDefense Protections Reference Guide.

Size of connection table:


* Use the fw tab -t connections command to find out the limit of the connections table.

* Use the fw tab -t connections -s command to find out the concurrent number of entries in the connections table.

NIC driver and NAPI:


In some cases, the NAPI string is added to the version name if the driver compiled with NAPI support. In such cases, the ethtool -i ethX command indicates whether the driver uses NAPI. However, not all drivers behave in that way. If the version name does not contain the NAPI string refer to the driver documentation or source code to understand whether the driver uses NAPI.

Use of single or dual affinity:


* Clear traffic:

o With SecureXL - Dual affinity

o Without SecureXL - Single affinity


* VPN traffic:

o With SecureXL - Dual affinity

o Without SecureXL - Single affinity


The kernel ARP table is limited to 1024 entries. The limit is set by the net.ipv4.neigh.default.gc_thresh3 parameter.

Note: changing the gc_thresh3 parameter will probably require to change the gc_thresh2 and gc_thresh1 accordingly in order for the garbage collector to work properly and not to overload the machine with garbage collections.

To enlarge the ARP cache entry table on-the-fly, run:

#sysctl -w net.ipv4.neigh.default.gc_thresh3=4096
#sysctl -w net.ipv4.neigh.default.gc_thresh2=2048
#sysctl -w net.ipv4.neigh.default.gc_thresh1=1024

or

#echo > /proc/sys/net/ipv4/neigh/default/gc_thresh3 4096
#echo > /proc/sys/net/ipv4/neigh/default/gc_thresh2 2048
#echo > /proc/sys/net/ipv4/neigh/default/gc_thresh1 1024

To make these changes survive a reboot, modify the /etc/sysctl.conf file to include the following lines:

net.ipv4.neigh.default.gc_thresh1 = 1024
net.ipv4.neigh.default.gc_thresh2 = 2048
net.ipv4.neigh.default.gc_thresh3 = 4096

and reboot the machine.

After reboot, you can verify values maintained, by using the more command:

#more /proc/sys/net/ipv4/neigh/default/gc_thresh1
#more /proc/sys/net/ipv4/neigh/default/gc_thresh2
#more /proc/sys/net/ipv4/neigh/default/gc_thresh3

Applies To:

* This solution replaces sk33856.

Workaround

N/A