Resolving PXF "Processing Capacity Exceeded" Errors
Article ID: 426513
Updated On:
Products
Issue/Introduction
When executing queries against external tables using the Platform Extension Framework (PXF), the query fails with the following error: ERROR: 08000: PXF server error: PXF Server processing capacity exceeded.
This indicates that the PXF service on one or more segment hosts has reached its limit for concurrent request processing and cannot handle additional tasks.
The log snippet:
2026-01-22 08:36:45.388013 <timezone>,"<username>","<dbname>",<pid>,<thread id>,"<IP Address>","57500",2026-01-22 08:36:45 <timezone>,0,<conid>,
<cmdid>,<segid>,<sliceid>,<transaction id>,,sx1,"ERROR","08000","PXF server error : PXF Server processing capacity exceeded.",,"Consider increasing the values of
'pxf.task.pool.max-size' and/or 'pxf.task.pool.queue-capacity' in '/usr/local/pxf-gp6/conf/pxf-application.properties' Check the PXF logs located
in the '/usr/local/pxf-gp6/logs' directory on host 'localhost' or 'set client_min_messages=LOG' for additional details.",,,
Environment
Greenplum 6 + PXF 6
Cause
The error occurs when the volume of concurrent PXF requests exceeds the allocated thread pool or queue capacity. In high-concurrency environments, the default settings may be insufficient to handle the number of "slices" or parallel connections initiated by Greenplum.
A secondary contributing factor can be network instability (e.g., UnknownHostException). If network requests hang or time out slowly, PXF threads remain occupied longer than expected, quickly exhausting the available thread pool.
Resolution
To resolve this, as advised in the log itself, you might want to tune the thread pool parameters and ensure sufficient JVM memory is available.
1. Modify PXF Configuration
Edit the pxf-application.properties file (typically located in /usr/local/pxf-gp6/conf/) on the host and adjust the following:
| Parameter | Recommended Action | Purpose |
pxf.task.pool.max-size |
Defult 200, increase to 300 - 400 | Increases internal worker threads for data tasks. |
pxf.task.pool.queue-capacity |
Default 0, set to 50 - 100 | Provides a buffer for requests when all threads are busy. |
pxf.max.threads |
Default 200, set >= max-size |
Ensures the underlying Tomcat server can accept the incoming requests. |
2. Adjust JVM Memory (Critical)
Increasing threads requires more memory. Edit pxf-env.sh to increase the Heap size to prevent OutOfMemoryError:
- Set
PXF_JVM_OPTS="-Xmx4g -Xms4g"(or higher depending on available system RAM).
3. Apply Changes
Run the following commands to synchronize and restart the PXF cluster:
pxf cluster sync
pxf cluster restart
4. Address Network Root Causes
Check for intermittent DNS or connectivity issues. If UnknownHostException appears in the logs, resolve the network resolution issues for your DataLake to prevent thread saturation.
Feedback
