Rollup data calculation
search cancel

Rollup data calculation

book

Article ID: 206586

calendar_today

Updated On:

Products

CA Infrastructure Management

Issue/Introduction

Questions about forcing and recalculation of rollups when managing CAMM and SNMP data for the same device in DX NetOps Performance Management.

We know that the rollups would not be calculated if the "high-watermark" for the Metric Family & CAMM data was past the timestamp of the data added to the database.

In the environment that raises these questions, we see that rollups get calculated even if the data is a few hours "late".

Data is imported using a custom CAMM pack into the Interface Metric Family. This Metric Family is also getting data from SNMP devices so the high-watermark should always be current.

The CAMM data tied to the same Metric Family is typically at least 2 hours behind but could be delayed more due to the various components involved.

Is there a way to know what delay is allowed?

For example, are hourly rollups always calculated X hours after their timestamp?

Are daily rollups calculated at a specific time of the day?

Environment

All supported DX NetOps Performance Management releases

Cause

Need to ensure methods are known to recalculate rollups when required.

Resolution

What happens here will depend on where the metrics are gathered and managed.

  • Does one Data Collector manage the data for both CAMM and SNMP for a single device?
  • Does one Data Collector manage the data for CAMM and one manages the SNMP data for a single device?

SNMP and CAMM on different Data Collectors is the only way this can work.

  • SNMP rollups run when the next bottom of the hour comes for DC ID X, and calculates the previous hour (which is higher than high water mark for DC X and ifstats_ltd).
  • CAMM rollups run when the next bottom of the hour comes for DC ID Y, and calculates the next rollup using data after the high water mark for DC Y and ifstats_ltd.
  • If CAMM is delayed getting data into DA by 2 hrs, As-Polled will be 2 hrs behind as well as rollups.
  • The issue is if CAMM and SNMP are coming from the same DC, then the SNMP rollups will cause the CAMM rollups to not run when it's data comes in.
  • You could have some SNMP devices on the CAMM DC and many not. The ones not on the CAMM DC will work and ones on the CAMM DC don't rollup CAMM data.
  • For checking the rollup high water marks can review this page:
    • http://DAHOST:8581/debug/dispatcher/container/com.ca.im.aggregator.loader/bean/rollupFactoryImpl

How does the reconciliation/isolation work between SNMP vs CAMM items in the following scenarios?

  1. SNMP device polled by DC#1, IPDomain “A”. CAMM provides the other components for the same device, on DC#1, IPDomain “A”. Will the CAMM components be created for the same device and data collected?
  2. SNMP device polled by DC#1, IPDomain “A”. CAMM provides the other components for the same device, on DC#2, IPDomain “A”. Will the CAMM components be created for the same device and data collected?
  3. SNMP device polled by DC#1, IPDomain “A”. CAMM provides the other components for the same device, on DC#2, IPDomain “B”. Will the CAMM components be created for the same device and data collected?

What about the same question for these scenarios?

  1. SNMP device polled by DC#1, IPDomain “A”. CAMM provides the same type of components (like additional interfaces) for the same device, on DC#1, IPDomain “A”. Will the CAMM components be created for the same device and data collected?
  2. SNMP device polled by DC#1, IPDomain “A”. CAMM provides the same type of components (like additional interfaces) for the same device, on DC#2, IPDomain “A”. Will the CAMM components be created for the same device and data collected?
  3. SNMP device polled by DC#1, IPDomain “A”. CAMM provides the same type of components (like additional interfaces) for the same device, on DC#2, IPDomain “B”. Will the CAMM components be created for the same device and data collected?

Both sets of scenarios result in the same answers:

  1. SNMP and CAMM items and PCC created on DC#1
  2. SNMP items and PCC created on DC#1, CAMM items created on same device, but PCC is on DC#2.
  3. SNMP items and PCC created on DC#1, CAMM items created as NEW device since diff IP Domain with PCC/items on DC#2

Note that when it comes to rollups:

  • With different MF rollups run fine. But same MF between CAMM/SNMP will process the rollups of the first SNMP or CAMM to provide data.
  • With different or the same MF rollups run fine since they have different DCM ids.
  • It doesn't matter if shared or same MF, 2 different IP domains so rollups run fine.

Only issue is #1 when the SNMP/CAMM are on same IP Domain and DC.

Additional Information

If working with an environment configured like this, ensure the data consumers understand that if there is a gap in the middle of data because a system was unavailable, it is possible to go back via the CAMM pack to get the missing raw data.

Calculating the rollup is another complexity. It would work if all data after a specific timestamp was dropped, the high-watermark for the MF was cleared and the CAMM pack was run to re-import the data sequentially.

Keep in mind this also means deleting the rows in ifstats_ltd/eqd/gtd/rate for the CAMM item_ids to reload data to for tstamp >= beginning of the data gap.

Powered by Wolken Software