Missing data points for random metrics in Odata output requested every 5 minutes

Products

• CA Infrastructure Management
• CA Performance Management - Usage and Administration

Issue/Introduction

Using OpenAPI to collect data on various devices and components and have experienced instances of missing data in the query results.

Running a query for metrics every 5 minutes for the last 5 minutes worth of data.

Environment

All supported Performance Management releases

Cause

Query frequency is too frequent. It doesn't allow enough time for the polled data to be inserted to the database after each poll cycle completes.

As a result, it's random what metrics will have had the data inserted when they query is resulting in mixed results.

The problem stems from the inability to know exactly when a poll is run or started, or when the data point is inserted to the DB for retrieval. No way to determine and base a schedule off that timing.

Resolution

Increase the query frequency so it's issued every 7-8 minutes for improved results.

Additional Information

• If the data is there in the DB as seen in Last Hour reports for the same metric and data, via the PM web UI reports, it validates there is no missing poll data.
• The intent of the OpenAPI tool is not for gathering "live" data.
• We don't partially load certain metrics for a given item. We get all the metrics in a Metric Family for one item at a time. Then we save them to a file and when the 400 MB size or 5 minute timer hits, we load it to the DB. As a result it is possible not all the items have provided their metric data before we load the file to the DB. If the query hits at the right time there will be metric data for some items and not others.