Portal and DA Group membership don't match

Products

• Network Observability
• CA Performance Management

Issue/Introduction

Threshold Violation Event raised against item not associated to Event Rule

Interface item that isn't a member of the reports context group still appears in Dashboard Views and other Portal Reports.

A group is created containing interface items. Tunnel interfaces with Tu in the name are excluded from the group membership.

The Portal Manage Groups page shows the Group. The Group shows all expected items and does not include the unwanted Tunnel interfaces.

Despite this we're seeing a Threshold Violation Event raise and clear. The raise happens every once in a while, when the tunnel interface violates the rule threshold.

The clear happens at 3 AM following the event raise. The clear event states the clear reason as:

Reason for clearing: This item is no longer associated with the profile that raised the Threshold Violation.)

Running the following vsql against the DR DB database, using the problem items DA Item_ID where <DA_ItemID> is, shows the DA still thinks the item is a member of the group.

select * from dauser.vgroup where group_item_id IN (select group_item_id from dauser.v_group_membership where member_item_id=<DA_ItemID>) ;

 

This issue can also cause Dashboard Views or other Reports to include the interface where it shouldn't be seen. The same incorrect Group membership on the DA can be observed through report runs as well.

Environment

All supported Network Observability DX NetOps Performance Management releases

Cause

The Data Aggregator (DA) still sees the item as a member of the group while the Portal does not.

Resolution

To resolve the issue we manually add/remove the item to force a DA DS incremental sync to add then remove the item. This should result in fixed DA group membership which will prevent further unwanted events from the item.

Steps to follow to resolve:

1. Manually add the item to the group via Portal UI.
   • Go to the Manage Groups page; edit the group; add the problem Tu interface and save the changes.
   • Do it manually instead of using group rule edits.
   • Less risky this way. It ensures only the problem item is moved around.
2. After adding it to the group membership via the Portal UI, wait 30 minutes.
   • Ensures a complete DA DS incremental sync is run.
3. Manually remove the item from the group via Portal UI.
   • Go to the Manage Groups page; edit the group and remove the problem Tu interface from the membership. Save the changes.
4. After removing it from the group membership via the Portal UI, wait 30 minutes.
   • Ensures a complete DA DS incremental sync is run.

To track if it's fixed or not run this on the DR DB in vsql. The member_item_id is the problem interface we see Events against. The is resolved when we no longer see the item as a member of the group in the DR DB.

• select * from dauser.vgroup where group_item_id IN (select group_item_id from dauser.v_group_membership where member_item_id=<DA_ItemID>) ;

Run the vsql before making changes, and again after completing the fix.