PXF Out Of Memory Issue - GC overhead limit exceeded

Products

• VMware Tanzu Greenplum

Issue/Introduction

Out of Memory (OOM) issue on the PXF can cause pxf to go down on one or more hosts:

java.lang.OutOfMemoryError: GC overhead limit exceeded
        at org.greenplum.pxf.api.io.GPDBWritable.setString(GPDBWritable.java:581)
        at org.greenplum.pxf.service.BridgeOutputBuilder.fillOneGPDBWritableField(BridgeOutputBuilder.java:412)
        at org.greenplum.pxf.service.BridgeOutputBuilder.fillGPDBWritable(BridgeOutputBuilder.java:238)

Resolution

• Open the pxf-env.sh file under $PXF_BASE/conf directory on the master/coordinator (here PXF_BASE is the user configuration directory that was created during initialisation of PXF. Yopu might be using the default directory $PXF_HOME/conf)
• Edit the pxf-env.sh file
• Change the Param PXF_JVM_OPTS.  The default value for this is '-Xmx2g -Xms1g'
• Change this value to the desired memory setting. Example:

  PXF_JVM_OPTS="-Xmx3g -Xms3g"

• Save the file
• Run pxf cluster sync to to copy the updated pxf-env.sh file to all the hosts in the cluster
• Restart PXF

See also Memory and Threading and Troubleshooting guide in the documentation for more details.

The option "-Xmx" sets the maximum memory the PXF java process is allowed to use. "-Xmx3g" sets the maximum to be 3GB.

The option "-Xms" sets the starting memory the PXF java process takes when it starts up. "-Xms3g" sets it to 3GB.