Optimizing CPU core assignments for 64-bit Check Point applications on the APM-8650
Article ID: 167919
Updated On:
Products
XOS
Issue/Introduction
This article describes CPU core mapping on the eight-core APM-8650 when Check Point R75.40VS (in Security Gateway mode), Check Point Security Gateway R75.40, Check Point Security Gateway R70-HCC, or Check Point Firewall-1 GX 5.0 is installed, and provides suggestions for configuring core affinities to optimize Switched Data Path (SDP) and firewall processing.When a 64-bit Check Point application is installed on the eight-core APM-8650 with CoreXL enabled, the default automatic sim affinity setting distributes SDPs across the cores in a way that may result in sub-optimal performance.
Cause
Problem:
When Check Point R75.40VS (in Security Gateway mode) or Check Point Security Gateway R75.40 is installed and Performance Pack and CoreXL are enabled, the default automatic sim affinity setting distributes SDPs to all cores, including cores running firewall instances.
When Check Point Security Gateway R70-HCC or Check Point Firewall-1 GX 5.0 is installed and Performance Pack and CoreXL are enabled, the default automatic sim affinity setting distributes SDPs unevenly to the available cores.
In all cases, the default automatic sim affinity setting fails to redistribute SDPs to available cores dynamically.
On 64-bit systems, the assignment of CPU#s (cores) to the physical processors (sockets 0 and 1) is different than on 32-bit systems. On 64-bit systems, cores CPU# 0, 2, 4, and 6 are assigned to socket# 0 and cores CPU# 1, 3, 5, and 7 are assigned to socket# 1. In contrast, on 32-bit systems, the CPU#s are assigned sequentially, with cores CPU# 0, 1, 2, and 3 assigned to socket# 0, and CPU# 4, 5, 6, and 7 assigned to socket# 1.
When CoreXL is enabled for a 64-bit Check Point application, firewall instances are assigned to cores sequentially, starting with the core with the highest CPU# on socket# 1, then to the remaining cores on socket# 1. If four firewall instances are enabled, they are assigned to cores CPU# 7, 5, 3, and 1 on socket# 1. If six firewall instances are enabled, the first four instances are assigned to cores CPU# 7, 5, 3, and 1 on socket# 1, and the last two instances are assigned to cores CPU# 6 and 4 on socket# 0.
The following example shows the eight-core APM-8650 with R75.40 or R75.40VS installed, four firewall instances enabled, and the 64-bit default SDP core mappings. Both a firewall instance and an SDP run on each of the cores CPU# 7, 5, 3, and 1, which can result less than optimal performance.
When Check Point R75.40VS (in Security Gateway mode) or Check Point Security Gateway R75.40 is installed and Performance Pack and CoreXL are enabled, the default automatic sim affinity setting distributes SDPs to all cores, including cores running firewall instances.
When Check Point Security Gateway R70-HCC or Check Point Firewall-1 GX 5.0 is installed and Performance Pack and CoreXL are enabled, the default automatic sim affinity setting distributes SDPs unevenly to the available cores.
In all cases, the default automatic sim affinity setting fails to redistribute SDPs to available cores dynamically.
On 64-bit systems, the assignment of CPU#s (cores) to the physical processors (sockets 0 and 1) is different than on 32-bit systems. On 64-bit systems, cores CPU# 0, 2, 4, and 6 are assigned to socket# 0 and cores CPU# 1, 3, 5, and 7 are assigned to socket# 1. In contrast, on 32-bit systems, the CPU#s are assigned sequentially, with cores CPU# 0, 1, 2, and 3 assigned to socket# 0, and CPU# 4, 5, 6, and 7 assigned to socket# 1.
When CoreXL is enabled for a 64-bit Check Point application, firewall instances are assigned to cores sequentially, starting with the core with the highest CPU# on socket# 1, then to the remaining cores on socket# 1. If four firewall instances are enabled, they are assigned to cores CPU# 7, 5, 3, and 1 on socket# 1. If six firewall instances are enabled, the first four instances are assigned to cores CPU# 7, 5, 3, and 1 on socket# 1, and the last two instances are assigned to cores CPU# 6 and 4 on socket# 0.
The following example shows the eight-core APM-8650 with R75.40 or R75.40VS installed, four firewall instances enabled, and the 64-bit default SDP core mappings. Both a firewall instance and an SDP run on each of the cores CPU# 7, 5, 3, and 1, which can result less than optimal performance.
<VAP_group_name>_<index> (test): ~# fw ctl affinity -l -r -v
CPU 0: sdp0 (irq 153)
CPU 1: sdp1 (irq 161)
fw_3
CPU 2: sdp2 (irq 177)
CPU 3: sdp3 (irq 201)
fw_2
CPU 4: sdp4 (irq 169)
CPU 5: sdp5 (irq 185)
fw_1
CPU 6: sdp6 (irq 193)
CPU 7: sdp7 (irq 145)
fw_0
All: eth1 (irq 90) eth0 (irq 98)
mpdaemon fwd cprid cpd
<VAP_group_name>_<index> (test): ~#
CPU 0: sdp0 (irq 153)
CPU 1: sdp1 (irq 161)
fw_3
CPU 2: sdp2 (irq 177)
CPU 3: sdp3 (irq 201)
fw_2
CPU 4: sdp4 (irq 169)
CPU 5: sdp5 (irq 185)
fw_1
CPU 6: sdp6 (irq 193)
CPU 7: sdp7 (irq 145)
fw_0
All: eth1 (irq 90) eth0 (irq 98)
mpdaemon fwd cprid cpd
<VAP_group_name>_<index> (test): ~#
Resolution
To avoid possible performance issues, you must manually assign SDPs to the cores, following these basic principles:
2. Run the following command and set affinities so that the SDPs are evenly distributed across only those cores that do not run firewall instances.
3. Verify the configuration by running the following command.
Note:
If you have enabled six firewall instances, assign the SDPs only to CPU# 0 and CPU# 2. For example:
Verify the affinity settings.
This results in a configuration that assigns the firewall instances to CPU# 7, 5, 3, 1, 6, and 4 and assigns the SDPs to CPU# 2 and 0.
Note:
If you have a single-processor APM-8650 (four cores: CPU#s 0, 1, 2,and 3), Blue Coat recommends that you enable three firewall instances. Use the techniques described above to verify that the firewall instances run on the cores CPU# 1, 2, and 3, and then assign all SDPs to run on CPU# 0.
- Do not assign SDPs to the cores that are running firewall instances.
- Assign SDPs so they are distributed evenly across the cores on the same socket.
- The Check Point application correctly handles core assignments of firewall instances when you enable CoreXL. Blue Coat recommends that you do not change these settings manually.
To configure SDP core affinities
1. Determine which cores run firewall instances and which do not. To do this, open an rsh session on each VAP and run the following command.
<VAP_group_name>_<index> (test): ~# fw ctl affinity -l -r -v
CPU 0: sdp0 (irq 153)
CPU 1: sdp1 (irq 161)
fw_3
CPU 2: sdp2 (irq 177)
CPU 3: sdp3 (irq 201)
fw_2
CPU 4: sdp4 (irq 169)
CPU 5: sdp5 (irq 185)
fw_1
CPU 6: sdp6 (irq 193)
CPU 7: sdp7 (irq 145)
fw_0
All: eth1 (irq 90) eth0 (irq 98)
mpdaemon fwd cprid cpd
<VAP_group_name>_<index> (test): ~#
This example shows a sub-optimal configuration for four firewall instances. Notice that the cores CPU# 7, 5, 3, and 1 are each shared by a firewall instance and an SDP.
CPU 0: sdp0 (irq 153)
CPU 1: sdp1 (irq 161)
fw_3
CPU 2: sdp2 (irq 177)
CPU 3: sdp3 (irq 201)
fw_2
CPU 4: sdp4 (irq 169)
CPU 5: sdp5 (irq 185)
fw_1
CPU 6: sdp6 (irq 193)
CPU 7: sdp7 (irq 145)
fw_0
All: eth1 (irq 90) eth0 (irq 98)
mpdaemon fwd cprid cpd
<VAP_group_name>_<index> (test): ~#
This example shows a sub-optimal configuration for four firewall instances. Notice that the cores CPU# 7, 5, 3, and 1 are each shared by a firewall instance and an SDP.
2. Run the following command and set affinities so that the SDPs are evenly distributed across only those cores that do not run firewall instances.
<VAP_group_name>_<index> (test): ~# sim affinity -s
eth1 [All] :
eth0 [All] :
sdp0 [0] : 0
sdp1 [1] : 0
sdp2 [2] : 2
sdp3 [3] : 2
sdp4 [0] : 4
sdp5 [1] : 4
sdp6 [2] : 6
sdp7 [3] : 6
<VAP_group_name>_<index> (test): ~#
eth1 [All] :
eth0 [All] :
sdp0 [0] : 0
sdp1 [1] : 0
sdp2 [2] : 2
sdp3 [3] : 2
sdp4 [0] : 4
sdp5 [1] : 4
sdp6 [2] : 6
sdp7 [3] : 6
<VAP_group_name>_<index> (test): ~#
3. Verify the configuration by running the following command.
<VAP_group_name>_<index> (test): ~# fw ctl affinity -l -r –v
CPU 0: sdp0 (irq 153) sdp1 (irq 161)
CPU 1: fw_3
CPU 2: sdp2 (irq 177) sdp3 (irq 201)
CPU 3: fw_2
CPU 4: sdp4 (irq 169) sdp5 (irq 185)
CPU 5: fw_1
CPU 6: sdp6 (irq 193) sdp7 (irq 145)
CPU 7: fw_0
All: eth1 (irq 90) eth0 (irq 98)
mpdaemon in.asessiond vpnd in.geod fwd cprid cpd
<VAP_group_name>_<index> (test): ~#
CPU 0: sdp0 (irq 153) sdp1 (irq 161)
CPU 1: fw_3
CPU 2: sdp2 (irq 177) sdp3 (irq 201)
CPU 3: fw_2
CPU 4: sdp4 (irq 169) sdp5 (irq 185)
CPU 5: fw_1
CPU 6: sdp6 (irq 193) sdp7 (irq 145)
CPU 7: fw_0
All: eth1 (irq 90) eth0 (irq 98)
mpdaemon in.asessiond vpnd in.geod fwd cprid cpd
<VAP_group_name>_<index> (test): ~#
The result is a configuration that prevents an SDP and a firewall instance from sharing the same core. In this example, the configuration assigns the firewall instances to the socket# 1 cores CPU# 7, 5, 3, and 1 and assigns the SDPs to the socket# 0 cores CPU# 6, 4, 2, and 0.
Note:
If you have enabled six firewall instances, assign the SDPs only to CPU# 0 and CPU# 2. For example:
<VAP_group_name>_<index> (test): ~# sim affinity -s
eth1 [All] :
eth0 [All] :
sdp0 : 0
sdp1 : 0
sdp2 : 0
sdp3 : 0
sdp4 : 2
sdp5 : 2
sdp6 : 2
sdp7 : 2
<VAP_group_name>_<index> (test): ~#
eth1 [All] :
eth0 [All] :
sdp0 : 0
sdp1 : 0
sdp2 : 0
sdp3 : 0
sdp4 : 2
sdp5 : 2
sdp6 : 2
sdp7 : 2
<VAP_group_name>_<index> (test): ~#
Verify the affinity settings.
<VAP_group_name>_<index> (test): ~# fw ctl affinity -l -r –v
CPU 0: sdp0 (irq 153) sdp1 (irq 161) sdp2 (irq 177) sdp3 (irq 201)
CPU 1: fw_3
CPU 2: sdp4 (irq 169) sdp5 (irq 185) sdp6 (irq 193) sdp7 (irq 145)
CPU 3: fw_2
CPU 4: fw_5
CPU 5: fw_1
CPU 6: fw_4
CPU 7: fw_0
All: eth1 (irq 90) eth0 (irq 98)
mpdaemon in.asessiond vpnd in.geod fwd cprid cpd
<VAP_group_name>_<index> (test): ~#
CPU 0: sdp0 (irq 153) sdp1 (irq 161) sdp2 (irq 177) sdp3 (irq 201)
CPU 1: fw_3
CPU 2: sdp4 (irq 169) sdp5 (irq 185) sdp6 (irq 193) sdp7 (irq 145)
CPU 3: fw_2
CPU 4: fw_5
CPU 5: fw_1
CPU 6: fw_4
CPU 7: fw_0
All: eth1 (irq 90) eth0 (irq 98)
mpdaemon in.asessiond vpnd in.geod fwd cprid cpd
<VAP_group_name>_<index> (test): ~#
This results in a configuration that assigns the firewall instances to CPU# 7, 5, 3, 1, 6, and 4 and assigns the SDPs to CPU# 2 and 0.
Note:
If you have a single-processor APM-8650 (four cores: CPU#s 0, 1, 2,and 3), Blue Coat recommends that you enable three firewall instances. Use the techniques described above to verify that the firewall instances run on the cores CPU# 1, 2, and 3, and then assign all SDPs to run on CPU# 0.
Workaround
N/AFeedback
Yes
No
Powered by
