1. We've set up some metric groupings and associated alerts in DX APM SaaS for the first time. I have a few alerts whose warning condition has been met, but we've not received any emails. I've created Notification channels and enabled them. I've also sent test emails from DX APM to the email configured in the notification channel and received those each time. So not sure why we're not receiving emails when the alert conditions are being met.
2. We are still struggling with frequent alerts “storms” where 1400 alerts come in at one time reporting conditions as “normal”. Is there any way to suppress these in APM alert configuration? In the op-prem instance we had alerts configured to only alerts when “severity increases” and that’s when we received alerts for the impacted metric. We have same logic in SaaS versions but are frequently bombarded with thousands of email alerts informing condition is normal when the alert criteria was never breached for various metrics.
Release : SAAS
Component : APM SAAS ENTERPRISE
1. New SaaS fix released.
2. "Is there any way to suppress these in APM alert configuration?"
-- No. In SaaS the only way is via Policy filter. You can filter out Severity "Unknown"/"Information"
It is a currently known issue
This section lists the known issues for the current release.
Defect DE466013: APM Alert - Trigger Notifications Alert Choice Ignored
Valid for: DX APM SaaS 20.6
Symptom: This issue happens after you select a trigger alert notification when creating a new alert. DX APM does not apply the trigger correctly when the alert level changes. For example, you create an alert, and select the option to be notified When Severity Increases.
When the alert severity actually changes from Critical to Major to OK, DX APM should not send an alert email notification. However, the notification channel triggers, incorrectly sending an email notification when the alert severity is reduced.
Solution: There is no workaround for this issue.