Alert generated in Service Now ticket, but no longer seen in AAI
Article ID: 432127
Updated On:
Products
Issue/Introduction
When a jobstream run begins but is blocked by a predecessor failure, it may remain in an incomplete or unknown state․ If the target job does not run on its regular schedule to complete the flow, the jobstream run remains stalled․ After a set period, the system automatically deletes this incomplete run and its associated alerts, causing the Jobstream Run ID to disappear from the Dashboard and reports․
SYMPTOMS:
-
A jobstream alert email or snmp trap is generated generated for a breach, but the corresponding alert is missing from the AAI website for the Jobstream Run Id in the alert
-
The Jobstream Run ID cannot be found in the AAI Dashboard or run history reports
-
The jobstream appears to have started on one day but never officially completed
CONTEXT: This occurs when an upstream job fails, blocking the target job, and the jobstream remains incomplete for more than 72 hours․ IMPACT: Users may be confused by missing alerts or runs in the AAI Dashboard, despite receiving external incident tickets (e․g․, ServiceNow)․
Cause
The system's default UNKNOWN_JOBSTREAM_RUN_LOOKBACK parameter by default is set to 72 hours․ When a job stream run starts but is permanently blocked (e․g․, due to a predecessor job failure), it sits in an "unknown" state․ Once this 72-hour limit expires, the Jobstream Run and any associated alerts will get deleted from the database.
Resolution
1․ IDENTIFY THE MISSING RUN ID
Action: Review your external ticketing system (e․g․, ServiceNow) to find the specific Jobstream Run ID that triggered the alert․
EXPECTED: You have the missing Run ID․
2․ VERIFY TARGET JOB EXECUTION
Action: Check the execution history of the target job for the jobstream in question․
EXPECTED: The target job did not run or complete during the timeframe of the missing Jobstream Run ID․
3․ UNDERSTAND SYSTEM BEHAVIOR
Action: Recognize that this is working as designed․ AAI automatically cleans up stale data․ If a jobstream run remains in an incomplete, unknown state for longer than the configured threshold, the Database Maintenance process purges the run and its alerts․
Feedback
