Formulate and Apply global rate limit values per pod

Products

• CA API Gateway

Issue/Introduction

currently we are facing a issue during peak loads where the pod becomes overwhelmed with lot of request and with our given HPA, it takes time to ramp up the number of pods (optimization of HPA is a separate task and should not be considered as part of the ticket).

In such case we wish to apply a global rate limit i.e. irrespective of our policy, or client id requesting, but an overall value describing the number of requests a pod could handle and should reject anything over and above it.

Regarding this we need your inputs around:

1. What is the best way to handle such global rate limits at pod level.
2. How could we arrive to this magical number of best fit ceiling value for this rate limit.
3. In case we go with naive approach of checking it against the number of available threads and say apply 10x available thread limit as rate limit value, is there any global variable (like available threads for underlying tomcat) that can be accessed in say onMessageRecieved global policy to be applied using apply rate limit assertion  

Environment

11.2

Resolution

You can actually have the apply rate limit assertion used , and have 

limit each parameter set to "Gateway" :-  that is basically giving you the rate limiting option per Gateway nodes and then have the When limit exceeded parameter set to "Throttle" .