API Gateway: A larger JVM heap size in Gateway 10 compared to Gateway 9.4, causing performance concerns when garbage collection runs

Products

• CA API Gateway

Issue/Introduction

This article will discuss what can cause a larger JVM heap size in Gateway 10 versus the 9.4 series, and how to respond to it. A larger heap size means that the "cost of major garbage collections" goes up, effectively rendering the GC process less efficient and longer running. 

In Gateway version 10, we switched from UseParallelOldGC to the G1GC method for garbage collection. This brings about improved performance in the vast majority of cases, but there can be a negative impact in some rare circumstances. See the Cause section for more.

Environment

This article applies to API Gateway 10.x (or any appliances running with the G1GC method).

Cause

This usually occurs when the environment is handling large message sizes and auditing requests/responses, and other similar scenarios that means more data is saved and utilized for each request to the API Gateway.

Resolution

If the G1GC garbage collection method is causing performance bottlenecks or concerns due to the way the API Gateway is used in the environment, then the following steps should be followed to modify the behaviour:

1. Download the appliancedefs.sh attached to this article (it's from a 10.0 system but modified)
2. SCP this to your 10.0 test host(s).
3. Backup the original appliancedefs.sh file: cp /opt/SecureSpan/Gateway/runtime/etc/profile.d/appliancedefs.sh /opt/SecureSpan/Gateway/runtime/etc/profile.d/appliancedefs.orig
4. Copy the SCP file: cp /home/ssgconfig/appliancedefs.sh /opt/SecureSpan/Gateway/runtime/etc/profile.d/appliancedefs.sh
5. Modify the permissions and ownership of the file:
   1. chmod 555 /opt/SecureSpan/Gateway/runtime/etc/profile.d/appliancdefs.sh
   2. chown  layer7:layer7 /opt/SecureSpan/Gateway/runtime/etc/profile.d/appliancedefs.sh
6. Restart the API Gateway and validate the jvm usage.

Additional Information

The following was modified in the appliancedefs.sh file for reference:

# Setting larger permsize for java 1.6   <=== Exist in both versions 4 lines below are updated(3 are new 1 is now a comment).
# Remove new 10.0 GC Can cause issue in certain size messages
#NODE_OPTS="-Xss256k -XX:+UseG1GC -Xmx${java_ram}k -Xms${java_ram}k -XX:G1HeapRegionSize=${G1HeapRegionSize} -XX:+AlwaysPreTouch -XX:+ParallelRefProcEnabled -XX:+UnlockExperimentalVMOptions -XX:InitiatingHeapOccupancyPercent=70 -XX:G1NewSizePercent=20 -XX:G1MaxNewSizePercent=60"
# Add Old 9.4 GC Garbage Collection
NODE_OPTS="-Xmx${java_ram}k -Xss256k -XX:+UseParallelOldGC"