Latency and timeouts with Gateway 10.1 Cr01+

Products

• CA API Gateway

Issue/Introduction

If you see messages like below and / or experience latency and you are using OTK or http route assertions that utilize localhost.

When the gateway version starts 10.1 CR01, you might see the gateway throws the exception message (e.g., Connection reset) like below or the gateway experiences latency with response time greater than 60 seconds intermittently.

- com.l7tech.common.http.prov.apache.components.SsgDefaultHttpClient: I/O exception (java.net.SocketException) caught when connecting to the target host: Connection reset

Resolution

The issue has been addressed in Gateway v10.1.00 CR04 and Gateway v11.0.00 CR02.  

To resolve this issue, the newer version will have a cluster-wide property: "tomcat.nio.pollerThreadCount".

This property should be set to be greater than 1 (default value).  A reasonable value of the property depends on case by case.  In some  cases, 10 or 50 resolved the issue. So it may require setting the value at 10 and then adjusting it up until you get the setting that works the most efficiently in your environment. We suggest you perform your own tuning exercise and performance runs to make sure the value choosen is right for their load and environment.  It is not possible for us to provide a best generic value to apply as environments differ.
 

To get to cluster-wide properties: login to policy manager -> Tasks (on top menu) -> Global Settings -> Manage Cluster-Wide Properties 

In new dialog box click add.

The cluster-wide property description is "The number of threads allocated to handle polling events. This is a hard upper limit." and the default value of the property is 1.  If the property name adds a prefix, "com.l7tech.server.", it becomes a system property, which will be applied in a single gateway node.

After adding this ensure you restart the ssg service on the node(s) in the cluster.

 # service ssg restart