Alarms do not remain in maintenance for servers in maintenance
Article ID: 443461
Updated On:
Products
Issue/Introduction
You observe that alarms for devices in a maintenance window do not remain in maintenance mode. While the maintenance icon appears initially, subsequent alarm updates or a restart of the normalized alarm service cause the maintenance status to revert to "false." This issue typically occurs when maintenance schedules are older than the message retention period in the underlying messaging bus (Kafka), preventing the service from reloading the active schedules upon restart.
Environment
- Product: DX Operational Observability (DX O2)
- Version: 25.1, 26.1
Cause
The system reloads maintenance schedules from the message bus after a service restart. If the maintenance window was created before the retention period (typically 7 days), the schedule data is purged from the bus, and the service cannot re-verify the maintenance status for incoming alarm updates.
Resolution
This issue is confirmed as a defect (DE671941).
A fix is targeted to be included in a future hotfix for release 26.1. Contact Broadcom Support to verify the availability of this hotfix for your environment.
Workaround
If you cannot apply a hotfix immediately, you can use the following steps to resynchronize the maintenance schedules:
- Log in to the postgres database pod and clear the existing maintenance tables.
- Restart the normalized alarm pod to force a reload of current schedules.
Additional Information
Feedback
