No active nodes found. Limit specified for max request will be applied to each node instead.

book

Article ID: 224130

calendar_today

Updated On:

Products

CA API Gateway API SECURITY CA API Gateway CA API Gateway Enterprise Service Manager (Layer 7) CA Mobile API Gateway CA Rapid App Security STARTER PACK-7

Issue/Introduction

Error message seen for rate limit assertion:

 

11000 Policy processing caught an exception: Unexpected error while accessing provider. No active nodes found. Limit specified for max request will be applied to each node instead.

 

 

Cause

As per document,

https://techdocs.broadcom.com/us/en/ca-enterprise-software/layer7-api-management/api-gateway/10-1/policy-assertions/assertion-palette/service-availability-assertions/apply-rate-limit-assertion.html

"If you have a cluster of Gateways, the limits entered in this assertion are divided among the number of "up" nodes in the cluster. A node is considered “up” if it has posted its status within the past 8 seconds (configurable via the ratelimit.clusterStatusInterval cluster property)."

 

By default the cluster info provider is 'ssgdb', this can be found in /opt/SecureSpan/Gateway/node/default/etc/conf/system.properties file,

com.l7tech.server.extension.sharedClusterInfoProvider=ssgdb

 

Then the status timestamp of nodes can be queried on ssg database,

select name, FROM_UNIXTIME(statustimestamp/1000) statustime from cluster_info;

 

And compare to current time,

select now();

 

If the time difference is greater than 8 seconds, the nodes are considered as non-active nodes, for example,

If no active node found, it will cause the error message.

Environment

Release : 10.0

Component : API GATEWAY

Resolution

Increase the value of cluster-wide property ratelimit.clusterStatusInterval as per the time difference, for the above sample, double the value to 16000.

 

Besides, the gateway nodes by default should update its node status every 4 seconds, the update latency could be due to slow mysql. So, if it's confirmed that the mysql is in poor performance, another solution is to fix the mysql performance issue.

Additional Information

https://techdocs.broadcom.com/us/en/ca-enterprise-software/layer7-api-management/api-gateway/10-1/policy-assertions/assertion-palette/service-availability-assertions/apply-rate-limit-assertion.html

Attachments