SOI - Alarms do not respect Escalation Policy if they are on maintenance

Products

CA Service Operations Insight (SOI)

Issue/Introduction

We would like to exclude alarms from a particular service from being closed while being on maintenance using a criteria and matching a User Attribute previously added by an escalation policy rule.

According to escalation policy rules (https://techdocs.broadcom.com/us/en/ca-enterprise-software/it-operations-management/service-operations-insight-connectors/connectors/how-to-create-escalation-policy0.html) the policy assigned to service should be evaluated before the policy assigned to the alert queue.

We have a policy that adds a value (eg.NCC) to a user attribute (eg.9) for specific services:

And a policy to clear alerts in maintenance, which is assigned to alert queue with criteria maintenance equal true and user attribute 9 not equal NCC.

The problem is that the alert is cleared before the attribute is assigned if the service is in maintenance mode. (error from escalation policy history: Alert update failed - This alert has already been cleared.)

How can we achieve this?

Environment

Release : 4.2

Component : SOI ALERT MANAGEMENT

Cause

This is working as designed.

Flow of events:

• As soon as Alarm came in SOI, the service level escalation policy to update userAttribute with value "NCC" got executed correctly.
(verify with select * from alerts where mdralarmid='?' // replace ? with "Source Alert Id" or MDRElementId of alarm)
• The problem is that the alarm got assigned to the queue(with ClearAlarm policy) which is getting applied on that Alarm and alarm got cleared

Thi is happening because of following sequence of code flow:

1) SOI associate the alert with related CI and identifies that the alarm is on the CI in Maintenance
2) SOI checks the conditions of all alert queues and assigns the Alert to those matching queues. As per the problematic alert queue condition is Maintenance equals true and UserAttribute does not contain NCC, the new alert matches with these conditions and this queue(with ClearAlarm policy) gets assigned to that alert.
3) Apply Escalation policies where SOI checks and executes Service policy first correctly
4) As the queue got assigned(in #2 point above) to that new alert, the Clear Alarm policy gets applied

This condition is uncontrollable and no fix can be provided for it because queue assignation and escalation policies actions happens using 2 different code flows.

Resolution

A solution to this is using enriching the Alarms from the specific service with the needed userAttribute with value NCC by using event policy rather than Escalation policy and actions.

If the event policy is used by using some condition to update value NCC, the issue will be resolved.

Additional Information

To deploy an even policy to enrich alarms from the following steps can be followed.

The below steps needs to be performed to create and deploy the policy file:
NOTE: You need to follow the same steps to reduce the policy creation effort.

1) Create a new EventPolicy with name:(Eg.) "NCCPolicy" of "JDBC" type by providing correct DB details and deploy on either a specific connector or on MTC connector.
NOTE:: If MTC connector is selected to deploy the event policy, no other connector needs to be selected because that policy gets applied on alerts from ALL the connectors

2) Open following folders:
CA\SOI\resources\EventManagement\Policies : open file NCCPolicy.policy
CA\SOI\resources\Core\Catalogpolicy\extensions : Open file/files with name like *nccpolicy.xml

3) Search for <EventClass> tag and copy complete <EventClass> tag in notepad

4) Replace complete <EventClass> tag with the following in any one file above first and update correct DB details from the copied content in notepad:

NOTE:: Update the following in <EventClass> tag as required:
Update "TestService" with the service name on which you want to apply this policy
Update "userAttribute10" with the required user attribute which is used in the alert queue condition to apply clear alarm policy

5) Copy the above updated <EventClass> tag in all open files in #2 step above.

6) Save all files and Restart Integration Service

NOTE:: Eventpolicy is of strict type and case sensitive. If you want to create event policy file with different name, make sure that you are updating it correctly in <EventClass> tag. Any small mistake will take lots of time to debug and correct it.