During the Load and Performance Testing, we visibly noticed a performance degradation of the API GW Containerized 11.1.2 compared to the previous version 11.1.1.
When passing the 150 Requests per second limit on a Gateway pod, the response times deteriorated to over
- over 1 second on average on the 1st test run, and
- over 5 seconds on average on the 2nd test run.
What we could observe was log contention on StdOut and high times of Garbage Collection. We checked and can exclude CPU Throttling on the infrastructure side.
** Question: Which Garbage Collector is used by default? No additional settings have been applied *
Run #1 (14:40 - 15:10)
|
500 io.httpCoreConcurrency |
750 io.httpMaxConcurrency |
|
Heap Size |
8 Gi |
|
Memory (Request + Limits) |
12 Gi |
|
Requested CPU |
6 Cores |
|
Limit CPU |
10 Cores |
|
-XX:ParallelGCThreads |
4 |
|
Pods (HPA) |
40 - 45 |
Run #2 (16:50 - 17:40)
|
1200 io.httpCoreConcurrency |
1500 io.httpMaxConcurrency |
|
Heap Size |
12 Gi |
|
Memory (Request + Limits) |
16 Gi |
|
Requested CPU |
12 Cores |
|
Limit CPU |
12 Cores |
|
-XX:ParallelGCThreads |
8 |
|
Pods (HPA) |
20 |
API Container Gateway 11.1.2
Evaluate JSON Path Expression (V1) assertion" is depreciated, this causes a significant amount of logging check your console logs for the following entry:
1) The excessive logging of the deprecated assertion was responsible for the performance degradation.
2) Upgrade to v2 of the Evaluate JSON Path Assertion