Access gateway server intermittently down with JDK 11
Article ID: 424671
Updated On:
Products
Issue/Introduction
It is reported that Access gateway server httpd process was not responding to netscaler health prob intermittently, hence was marked "down".
Apache error_log:
[Wed MMM DD 12:44:05.235707 2026] [jk:warn] [pid 282385:tid 282385] No JkShmFile defined in httpd.conf. Using default ~/secure-proxy/httpd/logs/jk-runtime-status
[Wed MMM DD 12:44:05.237179 2026] [core:warn] [pid 282385:tid 282385] AH00098: pid file ~/secure-proxy/httpd/logs/httpd.pid overwritten -- Unclean shutdown of previous Apache run?
[Wed MMM DD 12:44:05.238924 2026] [mpm_worker:notice] [pid 282385:tid 282385] AH00292: Apache/2.4.63 (Unix) OpenSSL/3.4.0 mod_jk/1.2.50 configured -- resuming normal operations
[Wed MMM DD12:44:05.238976 2026] [core:notice] [pid 282385:tid 282385] AH00094: Command line: '~/secure-proxy/httpd/bin/httpd -d /~/secure-proxy/httpd -D SSL'
[Wed MMM DD 12:44:32.284797 2026] [mpm_worker:error] [pid 282385:tid 282385] AH00286: server reached MaxRequestWorkers setting, consider raising the MaxRequestWorkers setting
Environment
Access gateway: From 12.8.06
Cause
Httpd reached MaxRequestWorkers setting is just a symptom or effect.
There is actual hs_err_pid.log file indicated that JVM has crashed and code dump was generated.
# A fatal error has been detected by the Java Runtime Environment:
#
# SIGSEGV (0xb) at pc=0x00007f991f7e0bfa, pid=213286, tid=215639
#
# JRE version: OpenJDK Runtime Environment Temurin-11.0.28+6 (11.0.28+6) (build 11.0.28+6)
# Java VM: OpenJDK 64-Bit Server VM Temurin-11.0.28+6 (11.0.28+6, mixed mode, tiered, compressed oops, g1 gc, linux-amd64)
# Problematic frame:
# V [libjvm.so+0xd4bbfa] SATBMarkQueue::filter()+0x7a
#
# Core dump will be written. Default location: /var/crash/core.%e.213286.%h.%t
Resolution
The crash is caused by a **SIGSEGV (Segmentation Fault)** in the Java Virtual Machine's garbage collection subsystem, specifically in the G1GC (Garbage-First Garbage Collector) SATB (Snapshot-At-The-Beginning) marking queue filtering mechanism.
The crash occurred while the GC was filtering the SATB marking queue, attempting to dereference a pointer to an invalid memory region. This is an internal JVM error in the G1 garbage collector in OpenJDK 11.0.28 and is NOT caused by application code.
Try using a different garbage collector:
- Add "-XX:+UseParallelGC" to the JVM args
Steps to update the proxyserver.sh file:
1. edit the proxyserver.sh file (location: <secure-proxy>/proxy-engine)
2. Go to line JVM_PERF_OPTS="-server -XX:MaxMetaspaceSize=256M"
3. Add "-XX:+UseParallelGC"
Example: JVM_PERF_OPTS="-server -XX:MaxMetaspaceSize=256M -XX:+UseParallelGC"
4. Restart the Access Gateway Services.
Additional Information
Feedback
