OI SaaS Spectrum alarms not cleared

book

Article ID: 194023

calendar_today

Updated On:

Products

DX Operational Intelligence

Issue/Introduction

 
 
OI SaaS reports alerts but those alerts were already closed in Spectrum
 
 

Cause

 
 
Observed that all recent alarms are getting closed in alarms_all index. However some old spectrum alarms are in CLOSED state in alarms_spectrum index, but they are still in OPEN status. 
Due to threading issue alarm update ( service enrichment ) / alarm ingestion (as partial update from source product ) which comes at the same time, this can cause discarding one of the updates.
This was observed in other issues as well.

Environment

Release : 1.3.2

Component : CA DOI SPECTRUM DATA PUBLISHER

Resolution

 
 
 

To handle this problem, locking ( at cache in normalized alarm service )  has been handled, so that only one update will be handled at a time from cache entry.  

Has been deployed to OI SaaS environment