We are polling for Response Path Test ICMP items. The Attempts metric is showing a negative value once every hour. The following image shows a sample of the negative metric values seen in reports.
All supported Performance Management releases
Cisco device is not configured to maintain N number of prior bucketIndex values after a new one is generated.
Every hour a new bucketIndex is assigned to the tests to store their last hours worth of results. Normally when the bucketIndex changes the configuration on the device that controls the generation of the new bucketIndex values keeps N instances of the previous bucketIndex around before removing them.
Our code for these tests expects the bucketIndex to be kept around long enough to register the new bucketIndex value. At the end of the hour cycle when we poll with the old bucketIndex, the response should return with info showing it's changed along with the new one to use. We see the new value and update the item to be polled for the next hour with that new bucketIndex value.
Because it's being removed and no previous instances are there, we're getting a bad poll response every hour that results in a bad delta calculation which is seen as the negative value in reports.
The solution is to configure the devices to keep N instances of the bucketIndex around for these tests.
One instance might be sufficient, 2-3 might be better to provide a buffer. That would resolve this problem.