Aria Operations integration with Amazon AWS adapter stops collecting data

Products

• VMware Aria Suite

Issue/Introduction

Aria operations Amazon AWS adapter stops collecting data due to the number of API calls exceeding 50 call per second.

Environment

Aria Operations 8.18

Cause

from the amazon AWS adapter logs we see repeated throttling issues:

2024-06-06T16:31:23,599+0000 ERROR [Collector worker thread 2] (108402) com.vmware.adapter3.amazonaws.AWSCloudwatchManager.getMetricDataCollectionResults - error while fetching metric data results java.util.concurrent.Exec
utionException: software.amazon.awssdk.services.cloudwatch.model.CloudWatchException: Request has been throttled. (Service: CloudWatch, Status Code: 400, Request ID: XXXXXX-XXXX-XXXX-XXXXX-XXXXX)

Resolution

The issue is resolved in Aria Operations 8.18.2, and and the separate Amazon AWS MP that will be released alongside

The changes to the data collection from Amazon AWS will include:

• Get the list of metrics 
• Batch the metrics into groups of 500
• Initiate the getMetricDataAPI calls for each batch for each of the regions
• Keep a count of the number of API calls. if the call count reaches 25, introduce a 1 second delay (Both the count and the delay in milliseconds are configurable via the amazonaws.properties file)
• Fetch the results for the completed batch and associate to the appropriate resource.
• Reset the counter and  continue with the next batch of API calls for the given region 

This ensures that the call count per second does not exceed 50 API call per second.