Getting 100% memory swap error in one node of 4 in production

Products

• CA API Gateway

Issue/Introduction

We are getting - CRITICAL: Swap usage was 100.00 % (Used: 2.00 GiB, Free: 0.00 GiB, Total: 2.00 GiB)

This error in only one node that is also running master SSG DB 

 

Environment

Release : 10.1

Component :

Cause

TOP shows all gateways running between 13GB and 16GB memory, the gateway with master SSG DB will always run a little higher 

Before a major Garbage collection (GC) occurs on the primary Gateway total memory runs over 16GB close to 2GB, since SWAP size is 2 GB this will result in alerts (not a critical event but can be addressed)

 

Resolution

For the node that host the master SSG DB use 5/8 instead of 3/4 as multiplier so it would not take as much of the system memory 

Not recommended to increase the gateway memory on the gateway hosting SSG DB to 3/4

NOTE:  Other nodes can be set to 3/4

# vi /opt/SecureSpan/Gateway/runtime/etc/profile.d/appliancedefs.sh

Locate multiplier

system_ram=`grep MemTotal /proc/meminfo | awk '{print $2}'`

# Maximum amount of RAM to use

# ram units in Kilobytes

multiplier="1/2"

let java_ram="$system_ram*$multiplier"

Change 

multiplier="5/8"

After this change a restart of the gateway is required 

 

Another way to reduce the swap is increase the java heap size: 

To set the heap size on Software Gateways, edit the file node.properties and change the following setting:

node.java.heap=<value>

where "<value>" is the memory, in megabytes. For example, “node.java.heap=4096” sets the heap size to 4GB.

After edit the file, restart gagteway. 

Note:

 Keep the JVM heap size to less than half of your computer's RAM, unless you have very specific reasons to do otherwise.