APM alert email actions sending caution alerts when no action is specified
Article ID: 217713
Updated On:
Products
Issue/Introduction
Our customer has an APM 10.7 and they are trying to configure an alert email action that doesn't seem to behave as expected.
The scope is to create an email action that only alerts for danger NOT caution and disable the caution all together (so they only want to see danger alerts) in a non-individual manner meaning to receive an email with details of all metrics in one and not separate email for each.
Below we have replicated the issue from our side and the screen shot shows the configuration we are using a simple alert that contains 4 metrics and should not alert by individual metric, it is when status changes so we want an indication both when the metrics cross the danger and go back to normal state.
Unfortunately the alert notification is not behaving as expected it is warning for both caution and danger threshold where only the danger has an action.
Environment
Release : 10.7.0
Component : Integration with APM
Resolution
When using console alerts for example with the settings you get a notification that when danger is opened or when danger is closed i.e.
6/16/21 6:22:15 AM EDT
The Alert "RBS1" opened a danger alert with a target of 60:
SuperDomain|ibnbld004162|EPAgentProcess|EPAgent_StringTest|MIKE|URLS|/myurl/donkey:f = 96
SuperDomain|ibnbld004162|EPAgentProcess|EPAgent_StringTest|MIKE|URLS|/myurl/donkey:g = 96
6/16/21 6:23:15 AM EDT
The Alert "RBS1" closed a danger alert with a target of 60:
SuperDomain|ibnbld004162|EPAgentProcess|EPAgent_StringTest|MIKE|URLS|/myurl/donkey:f = 50
SuperDomain|ibnbld004162|EPAgentProcess|EPAgent_StringTest|MIKE|URLS|/myurl/donkey:g = 50
SuperDomain|ibnbld004162|EPAgentProcess|EPAgent_StringTest|MIKE|URLS|/myurl/donkey:d = 2
The email however seems to send the notification indicating the state of the alert at the time the status changes so instead of showing the danger closed it shows that the caution is present
Alert Name: RBS1
Time Triggered: 6/16/21 6:22:15 AM EDT
Alert Status: Danger
Problem Details:
Alert Name: RBS1
Time Triggered: 6/16/21 6:23:15 AM EDT
Alert Status: Caution
Problem Details:
So as Caution is of no concern to you then set that to the same value as danger
And then instead of showing caution it will report Normal
Alert Name: RBS1
Time Triggered: 6/16/21 6:32:45 AM EDT
Alert Status: Danger
Problem Details:
Alert Name: RBS1
Time Triggered: 6/16/21 6:33:15 AM EDT
Alert Status: Normal
Problem Details:
in line with corresponding opened an closed console danger alerts
6/16/21 6:32:45 AM EDT
The Alert "RBS1" opened a danger alert with a target of 60:
SuperDomain|ibnbld004162|EPAgentProcess|EPAgent_StringTest|MIKE|URLS|/myurl/donkey:f = 96
SuperDomain|ibnbld004162|EPAgentProcess|EPAgent_StringTest|MIKE|URLS|/myurl/donkey:g = 96
6/16/21 6:33:15 AM EDT
The Alert "RBS1" closed a danger alert with a target of 60:
SuperDomain|ibnbld004162|EPAgentProcess|EPAgent_StringTest|MIKE|URLS|/myurl/donkey:f = 50
SuperDomain|ibnbld004162|EPAgentProcess|EPAgent_StringTest|MIKE|URLS|/myurl/donkey:g = 50
SuperDomain|ibnbld004162|EPAgentProcess|EPAgent_StringTest|MIKE|URLS|/myurl/donkey:d = 2
If you only want to know when it is Danger then change the trigger to when severity increases
Feedback
