Threshold Events missing for CPU or Memory items in Performance Center
Article ID: 205873
Updated On:
Products
Issue/Introduction
Threshold Profile for CPU and Memory from Check Point firewalls is no longer alerting. Usage on some devices is high enough that an alert should have triggered. I even re-created the Device Collection, the Threshold Profile, and lowered the Threshold Violation numbers and Events are still not triggering.
Other thresholds for Interface metrics are still triggering Events.
Component Item Events are not seen in the Performance Center web UI in DX NetOps Performance Management.
We have synchronized the Component Items to Performance Center in order add them to Groups as unique reportable items. This was done following the instructions in the Knowledge Base Article titled How to synchronize unsynchronized Component element types to CA Performance Center.
After this only newly added CPU or Memory Component Items are raising Events in Performance Center.
Environment
All supported DX NetOps Performance Management releases
Cause
Logging wasn't able to be gathered in time to validate the suspected cause. It was likely that there was Inventory Synchronization timeout between the Event Manager and Device Manager services during an Event Manager Data Source synchronization cycle.
When the Event Manager Data Source built and populated the em.event_item_info table it did not have information about the Device<->Component Item relationship.
As a result it set the ParentItemID in em.event_item_info using the same ItemID used by the Component Item itself.
Due to this any queries using the Device ItemID when looking for the Component Items would never match the ItemID or ParentItemID column values in the em.event_item_info table.
This is what is done during the Event View generation in the PC UI and during Spectrum EM Event polls.
Once set these values are not updated during future synchronization cycles. A Component Item doesn't change it's Parent Device so there shouldn't a need to update an existing relationship under normal conditions.
To see the problem we'd run the following MySql query against the em database. Replace <ItemID> with the ItemID of the Component Item involved. We should see that ItemID value set for both the ItemID and the ParentItemID values.
select * from event_item_info where ItemID=<ItemID>;
Resolution
To resolve this we need to trigger a rebuild of the em.event_item_info table. To do so we'd need to run the following against the PC MySql EM DB general table. This sets the GroupPathCache.Reload and GroupPathCache.RebuildEventItemInfo attributes in the em.general table from default 'false' to 'true'. At the next incremental synchronization cycle for the Event Manager Data Source it will read the true values and rebuild the table. This will correct the Component Item entries setting them with the correct ParentItemID for their Parent Device Item.
update em.general set value='true' where attribute in ('GroupPathCache.Reload','GroupPathCache.RebuildEventItemInfo');
After the Event Manager Data Source sychronization completes and rebuilds the table, it will reset the GroupPathCache.Reload and GroupPathCache.RebuildEventItemInfo attribute values back to the default 'false'.
To see the problem is now resolved after an Event Manager Data Source synchronization completes, run the following MySql query against the em database. Replace <ItemID> with the ItemID of the Component Item involved. We should see the ItemID and ParentItemID values are now different.
select * from event_item_info where ItemID=<ItemID>;
Only new Events for the Component Items, ones raised new after this fix is implemented, will now show in the Performance Center web UI.
Feedback
