Internal error 500 received when requesting high volume of data from SQL rest API

Products

• CA Application Performance Management SaaS
• CA Application Performance Management (APM / Wily / Introscope)
• CA Application Performance Management Agent (APM / Wily / Introscope)

Issue/Introduction

Customer uses a call to SQL REST API via a curl command to retrieve all metrics received by the tenant in the last minute. 

The syntax used in the curl command is this:

curl -v -Lk -H "Authorization: Bearer $TOKEN" \-H "Accept: application/json" -H "Content-Type: application/json" \https://apmgw.dxi-na1.saas.broadcom.com/<tenant integer id>/apm/atc/api/apmData/query \-d "{ \"query\" : \"SELECT * FROM metric_data where ts>=$TIME \"}"

In this example, $TOKEN and $TIME are variables created to store the tenant token and the time frame.

When the number of rows returned by a call via curl to the SQL REST API exceeds some limit, an internal error is received:

We are getting the following error message:
"error":{"code":500,"message":"500,1000,-: INTERNAL_ERROR, -"}}

 

Environment

Release : Any

Component : Integration with APM

Cause

There is a limit in the SQL REST API.

Resolution

Engineering provided this syntax to access datastore directly. The output format is different but the information is the same.

POST /metrics/queryMetric
{
"querySpecifier": {
"op": "ALL"
},
"queryRange": {
// last 60 seconds
"rangeSize": 60 
},
// clamping the result (optional, but I would keep it)
"clampSize": 10000
}

The curl syntax would be as follows:

curl -v -Lk -H "Authorization: Bearer $TOKEN" \-H "Accept: application/json" -H "Content-Type: application/json" \https://apmgw.dxi-na1.saas.broadcom.com/metrics/queryMetric \-d "{ \"querySpecifier\": { \"op\": \"ALL\" }, \"queryRange\": { \"rangeSize\": 60 }, \"clampSize\": 10000 }"

Additional Information

clampSize is a failsafe that limits the amount of data to be retrieved in case a high number causes a timeout/error.

Please try avoiding frequent use of heavy queries such as in dashboards or periodic scripts.