understanding apm-resource-protection mitigation and overload-protection threshold configurations and expected impact

book

Article ID: 167801

calendar_today

Updated On:

Products

XOS

Issue/Introduction

Once the configured memory or cpu resource threshold is reached, the apm-resource-protection mitigates new flows for excessive hosts when mitigation threshold is reached, or all new flows when overload-threshold is reached. The mitigation is implemented for the entire vap-group and not just the vap-member (APM) reaching the defined threshold.
XOS configuration:
---
apm-resource-protection
  enable
  vap-group FW
    mitigation memory-threshold 80 95 cpu-threshold 80 95
    overload-protection memory-threshold 95 cpu-threshold 95

########

Traffic impact is observed, and log entries similar to the following are seen in /var/log/messages:
---
Mar 27 21:43:58 npm1 cbs_np6_flowd[556]: [I] [npm1 1.1.1.1] apm-rsrc-protect: vap-group FW (id 1): resource level above mitigation threshold: mitigating new flows from excessive hosts, (memory 38%, CPU 80%)
Mar 27 21:43:59 npm1 cbs_np6_flowd[556]: [I] [npm1 1.1.1.1] apm-rsrc-protect: vap-group FW (id 1): resource level above overload-protection threshold: discarding all new flows, (memory 38%, CPU 96%)
Mar 27 21:44:01 Pod10 cbshmonitord[4214]: [N] [Pod10 1.1.1.20] Violation (s=3, alarm) occurred 3 times: module:1, item:3655 (H_ID_ERP_PROT_VAP_OVERLOAD), time:"Thu Mar 27 21:43:59 2014"
Mar 27 21:44:02 Pod10 cbsalarmlogrd: AlarmID 5397 | Thu Mar 27 21:44:01 2014 | critical | np1 | flowProtectVapGroupOverload | Above VAP group overload threshold
Mar 27 21:44:04 Pod10 cbsalarmlogrd: AlarmID 5398 | Thu Mar 27 21:44:01 2014 | info | system | systemAlarm | New system alarm level (critical) 


#####

Cause

Provides details on APM resource protection mitigation and overload-protection threshold configurations and what to expect when a threshold is reached.

Resolution

APM resource protection feature allows to set the threshold limits for memory utilization and CPU utilization. As a best practice the mitigation threshold is set lower than the overload-protection threshold.
APM resource-protection monitors and identifies hosts i.e. IP addresses, that are generating excessive flow rate or excessive flow count, excluding the IP addresses configured manually within apm-resource-protection white-list
-----

Impact of reaching mitigation threshold:
When the resource level hits the mitigation threshold, apm-resource-protection prevents new flows to be created for excessive hosts for the vap-group FW.

As shown in the example above, once the cpu core utilization on a FW_1 vap-member hit the mitigation threshold of 80% utilization, the NPM started mitigating new flows from excessive hosts.
------------

Impact of reaching the overload-protection threshold:
Once the overload-protection threshold is reached, then the apm-resource-protection discards all new flows for the vap-group and not just excessive hosts.

In the example above, once the CPU core utilization reached the threshold of 95% utilization, the NPM started discarding all new flows for the vap-group FW.

##########


NOTE: The apm-resource-protection monitors the cpu utilization on all cores on all of the vap-group members, and if the threshold is reached on one of the cores on a single vap-member (APM), the apm-resource-protection mitigation or overload-protection takes affect for the entire vap-group.