ALERT: Some images may not load properly within the Knowledge Base Article. If you see a broken image, please right-click and select 'Open image in a new tab'. We apologize for this inconvenience.

Alarms are not closed in linked system after the APM metric stops reporting

book

Article ID: 226781

calendar_today

Updated On:

Products

DX Application Performance Management DX Operational Intelligence DX APM SaaS DX OI SaaS

Issue/Introduction

We are still seeing an open alert in our problem management system for a metric that is no longer reporting in DX APM.

 

When we check in the OI Alarms console there is no open alarm for the alert so why is this not closed in our problem system that receives alarm notifications from OI via a webhook channel?

 

Cause

If a metric for which an alert is triggered in APM stops reporting then the alert status changes to 0 - unknown.

This in turn is notified to OI which will close the open alarm and send out notifications on any appropriate channel.

Because the metric is no longer reporting the severity of the alarm is now unknown 

The defined webhook channel did not include the ${status} variable in the JSON payload and was adjusting ticket status based on the ${severity} value i.e closing the ticket when the severity returned to normal

 

 

Environment

Release : SAAS

Component :

Resolution

Include the Alarm Status - ${status} variable in the JSON payload and configure the receiving system to work against that field rather than the alarm severity when determining if an alarm is still open.

example output below

 

JSON payload options are listed here

Possible values for the ${status} variable are NEW, UPDATED and CLOSED

Attachments