Understand why alert is continuous firing and resolving (flapping) - DX OpenExplore
search cancel

Understand why alert is continuous firing and resolving (flapping) - DX OpenExplore

book

Article ID: 452091

calendar_today

Updated On:

Products

DX OpenExplore Observability

Issue/Introduction

Alerts in DX OpenExplore(DXOE)/Wavefront are continuously firing and resolving (flapping) over a period where the underlying metric condition remains continuously in a breach state.

  • Alerts trigger, auto-resolve, and re-trigger within short intervals (example, 1–2 minute gaps).

  • Alert history (post event) displays multiple firing events within a window where the metric was continuously breaching.

  • Live real-time alert evaluation shows missing or incomplete data points that are continuously being backfilled. 

Cause

The Alert Engine evaluates the alert condition based ONLY on the data points that have been received at the time of check.

Resolution

First identify your data shape, is there a continuous lag in data ingestion or was this caused by a temporary delay.

Shift the Evaluation Window - If data reporting is delayed, an alert checking decision might be made on a temporarily incomplete set of data values. 

Adjust your Alert Resolve Criteria - Configure the Resolve Window to ensures DXOE/Wavefront requires the condition to evaluate as false continuously for # minutes based on the data delay identified, before auto-resolving.

Additional Information

lag Function

default Function