Gemfire - How to increase the limit of the "socket-buffer-size" at the OS level on GemFire

Products

• VMware Tanzu Gemfire

Issue/Introduction

GemFire will log an info-level message explaining that the "socket-buffer-size” configured within GemFire is greater than the limit set by the OS. The instructions below will discuss how to raise the OS level limit to avoid seeing these messages and run with the desired socket buffer size.

[info 2018/03/05 17:28:57.556 EST Cluster1(8581)-server1 <P2P-Handshaker /10.112.179.21:0 Thread 3> tid=0x5b] Socket receive buffer size is 124,928 instead of the requested 512,000.

Environment

Cause

The default value of rmem_max and wmem_max are set to 124,928 in most Linux distributions. This might be enough for a low-latency general purpose network environment, or for applications such as DNS or Web Server. However, in GemFire, applications might request "socket-buffer-size” to set a value larger than 124,928 and increasing the limit at the OS-level will be necessary.

Resolution

To increase the OS level limit of "socket-buffer-size", follow the below steps:

1. Run the following command lines to check the default and max value of the receive or send buffer sizes:

To check the default and maximum amount for the receive socket memory, run the following commands:

$ cat /proc/sys/net/core/rmem_default
$ cat /proc/sys/net/core/rmem_max

To check the default and maximum amount for the send socket memory, run the following commands:

$ cat /proc/sys/net/core/wmem_default
$ cat /proc/sys/net/core/wmem_max

2. Change the following set of parameters as needed:

• /proc/sys/net/core/rmem_default > recv > 124928 > changed to 512000
• /proc/sys/net/core/wmem_default > send > 124928 > changed to 512000
• /proc/sys/net/core/rmem_max > 124928 > changed to 512000
• /proc/sys/net/core/wmem_max > 124928 > changed to 512000

3. In Red Hat Enterprise Linux systems, the default and maximum send buffer size and receive buffer size are set to 256 KB. 

# sysctl -w net.core.rmem_default=262144
# sysctl -w net.core.wmem_default=262144
# sysctl -w net.core.rmem_max=262144
# sysctl -w net.core.wmem_max=262144

To make the change permanent, add the following lines to the /etc/sysctl.conf file, which is used during the boot process:

net.core.rmem_default=512000
net.core.wmem_default=512000
net.core.rmem_max=512000
net.core.wmem_max=512000

Please refer to the following link for more details:

• https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/5/html/tuning_and_optimizing_red_hat_enterprise_linux_for_oracle_9i_and_10g_databases/sect-oracle_9i_and_10g_tuning_guide-adjusting_network_settings-changing_network_kernel_settings

Additional Information

https://knowledge.broadcom.com/external/article?articleNumber=294129.