Why is CPU utilization of policy evaluation in SGOS 6.5 much greater than in 6.2?

Products

• ProxySG Software - SGOS

Issue/Introduction

When the ProxySG appliance is upgraded to SGOS 6.5 from 6.2, customers report that 6.5's CPU utilization of policy evaluation is much greater than in 6.2.

In one instance, a customer reported that CPU utilization increased 20% and had TCP acceptance regulation (see Customer example).

This issue occurs mainly due to new policy features in SGOS 6.4 and nested policy (see Nested policy example). To support nested policy, the policy engine changed to combine many objects in one larger condition. If all objects in the condition are simple IP addresses or subnets, there is less impact on performance, but if the nested policy contains groups inside groups, the performance impact is greater. In addition, CPU utilization increases by a few percentage points after an upgrade is expected. 

Customer example

6.2.15.4:

=========

CPU 0                                               22%
    Policy evaluation - HTTP                         8%
        libpolicy_enforcement.so                     6%
    HTTP and FTP                                     8%
        libhttp.exe.so                               2%
    Access Logging                                   2%

    Miscellaneous                                    2%

    Object Store                                     1%

 

 

CPU 1                                               32%

    Policy evaluation - HTTP                         9%

        libpolicy_enforcement.so                     6%

    TCPIP                                            9%

        libstack.exe.so                              7%

    HTTP and FTP                                     9%

        libhttp.exe.so                               2%

        kernel.exe                                   1%

    Miscellaneous                                    2%

    Establishing new connections                     1%

    Object Store                                     1%

 

 

 

6.5.4.4:

========

CPU 0                                               33%  <<

    Policy evaluation - HTTP                        13%  <<

        libpolicy_enforcement.so                    10%  <<

    HTTP and FTP                                    11%

        libhttp.exe.so                               3%

        libutil.so                                   1%

        libtransactions.exe.so                       1%

        kernel.exe                                   1%

    Miscellaneous                                    3%

    Object Store                                     2%

        libce_admin.exe.so                           1%

    Establishing new connections                     1%

    Access Logging                                   1%

 

CPU 1                                               34%

    Policy evaluation - HTTP                        11%

        libpolicy_enforcement.so                     8%

    TCPIP                                            9%

        libstack.exe.so                              7%

    HTTP and FTP                                     9%

        libhttp.exe.so                               2%

        libtransactions.exe.so                       1%

        kernel.exe                                   1%

    Object Store                                     2%

        libce_admin.exe.so                           1%

    Miscellaneous                                    2%

    Access Logging                                   1%

 

This example result shows less than 45Mbps traffic. If the customer has much more traffic with nested group policy, “Policy evaluation – HTTP” on all of CPU would increase.

Nested policy example

<proxy>

ALLOW condition=__CondList1URL_2ch.net_Allow url.host.substring=2ch.net

define condition __CondList1URL_2ch.net_Allow
client.address=Group05
client.address=Group07
client.address=Group12
client.address=Group13
client.address=Group14
client.address=Group19
client.address=Group20
client.address=Group22
client.address=Group23
client.address=Group27
client.address=Group32
client.address=Group38
client.address=Group81
client.address=Group90
client.address=Group91
client.address=Group96
client.address=Group97
client.address=Group98
client.address=Group99
client.address=Group101
client.address=Group127

Resolution

If you still see high CPU utilization after disabling policy coverage, it is needed to change policy not to use nested condition.

Workaround

Upgrade to SGOS 6.5.5.4 or later, and then change policy coverage settings to disabled. It will reduce CPU utilization. 

New CLI commands policy coverage and policy no coverage have been added to enable or disable the policy coverage settings in SGOS 6.5.5.4.