How to capture JVM Standard Output/Standard Error (STDOUT/STDERR) messages with GemFire.

Products

• VMware Tanzu Gemfire

Issue/Introduction

Occasionally, when trying to troubleshoot GemFire issues it is necessary to capture output coming from System.out.println statements added in custom code, or from the JVM itself (such as thread dumps or javax.net.debug logging). In GemFire 9.x, such logging can be lost when the member is started using gfsh (GemFire SHell). The solution is to have GemFire redirect such output to its log file. However, this output redirection is not done by default as it is not always desirable that GemFire "hijack" STDOUT or STDERR.

Environment

Product Version: 9.10

Resolution

Prior to GemFire 9.5, getting STDOUT and STDERR in the logs required setting two system parameters (note they *both* have to be set):

  -Dgemfire.OSProcess.DISABLE_OUTPUT_REDIRECTION=false
  -Dgemfire.OSProcess.ENABLE_OUTPUT_REDIRECTION=true

(Note: when starting GemFire processes with gfsh, system property defines such as these need to be prefixed with the "--J=" flag as with other JVM options.)

Starting with GemFire 9.5, the "--redirect-output" flag was added to do the same thing (redirect STDOUT and STDERR to the GemFire log). However, there is one important caveat. When a security log is configured, using the "security-log-file" GemFire property, STDOUT and STDERR will be redirected to this file rather than the general GemFire "log-file" and you will need to look there for the expected messages.