Alarm count mismatch due to devices in Maintenance in Spectrum

Products

• DX Operational Observability
• DX SaaS

Issue/Introduction

The alarm count between Spectrum and OI is out of sync. Currently showing 2023 alarms in OI and 1509 in Spectrum.

I have compared the alarms in OI to those in Spectrum and all devices affected are in Maintenance Mode in Spectrum.

Environment

• DX O2
• Spectrum Data Publisher

Cause

This condition can occurs when schedules were created more than 24 hours ago and:

a) normalizedalarm service is restarted

b) dxo2 onpremise upgrade

c) dxo2 services restart

Resolution

Enhancement Request #US1065315 has been created to address this issue in a next release.

Workaround:

Delete the maintenance schedules from postgres and restart SPUB.

1. Bash into postgres pod
run -> psql -U aopuser -d doi_db
2. delete from maintenance_window;
3. delete from maintenance_schedule;
4. Restart SPUB