Avoid Long GC Pauses Due to Paging (Linux) in GemFire

Products

• VMware Tanzu Gemfire

Issue/Introduction

The purpose of this article is to describe how to configure swappiness and why it is important to configure it correctly on a GemFire distributed system.

Resolution

Swapping, also known as paging, is the use of secondary storage to store and retrieve application data on disk for use in RAM memory. The Operating System, by default, may choose to swap out a process or some of its heap due to low usage even if it is not running low on RAM. Swappiness is performed in order to prepare the OS for eventual memory requests. Swappiness should be avoided for GemFire JVMs.

Swapping can have a significant negative impact on GemFire's performance and should be avoided. Often, swapping manifests itself as GemFire nodes being forced out of the distributed system due to long periods of unresponsiveness.

How to avoid swapping on a GemFire host:

There are two ways to avoid memory being written to disk by Linux:

1. Setting swappiness to 0.
2. Locking memory.

Setting swappiness to 0:

The swappiness setting on Linux is a value between 0 and 100, where higher values encourage more optimistic swapping. The default value is 60. For GemFire, the swappiness should always be disabled by setting it to 0. This way the offload from memory to disk is controlled by GemFire using overflow or eviction instead of being at the mercy of the OS.

To avoid swapping, ensure that sufficient RAM memory is available on the host and that the number of running processes is accounted for and do not consume all the available RAM. Tools such as top can be used to monitor swap rates.

To see the current swappiness value that is set, enter the following at the command prompt:

cat /proc/sys/vm/swappiness 

To set the swappiness permanently, modify the /etc/sysctl.conf file:

vm.swappiness=0 

Using memory locking:

Refer to the Locking Memory section of the User´s Guide for GemFire for how to avoid heap or off-heap memory being swapped out to disk by the OS.