G1SATBMarkQueueSet::filter core dump and 503 errors in CA Access Gateway (SPS)

Products

• SITEMINDER
• CA Single Sign On Secure Proxy Server (SiteMinder)
• CA Single Sign-On

Issue/Introduction

Running CA Access Gateway (SPS) randomly this one experience crashes and when that occurs, the browsers get the return code HTTP 503.

This happens randomly, and on all CA Access Gateway (SPS) in the Production environment.

The restart of the CA Access Gateway (SPS) solves the issue, but doesn't prevent to happening again.

The crash happens in the JVM:

Current thread (0x00007fa494005730):  GCTaskThread "GC Thread#2" [stack: 0x00007fa4999fb000,0x00007fa499afb000] [id=17960]

Stack: [0x00007fa4999fb000,0x00007fa499afb000],  sp=0x00007fa499af9c58,  free space=1019k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
V  [libjvm.so+0x7608e4]  G1SATBMarkQueueSet::filter(SATBMarkQueue&)+0x74
V  [libjvm.so+0xcd7974]  SATBMarkQueueSet::flush_queue(SATBMarkQueue&)+0x14
V  [libjvm.so+0x704136]  G1RemarkThreadsClosure::do_thread(Thread*)+0x36
V  [libjvm.so+0xe62fbd]  Threads::threads_do(ThreadClosure*)+0x4d
V  [libjvm.so+0x707ffe]  G1CMRemarkTask::work(unsigned int)+0x12e
V  [libjvm.so+0xf1688f]  GangWorker::loop()+0x5f
V  [libjvm.so+0xf168ef]
V  [libjvm.so+0xe655d8]  Thread::call_run()+0xa8
V  [libjvm.so+0xc26851]  thread_native_entry(Thread*)+0xe1

Environment

CA Access Gateway (SPS) 12.8SP8CR01 on RedHat 7;
OpenJDK 64-Bit Server VM Temurin-17.0.11+9;

Dynatrace agent is configured;

Resolution

Apply the 3 changes to the environment to progress to get more information about the JVM.

1. Update the JDK to
   
   JDK 17.17(Temurin 17.0.17+10 - 10/26/2025)
   
   
2. Isolate the Dynatrace Agent:
   
   Disable Dynatrace Agent on a CA Access Gateway (SPS) instance;
   
   
3. Switch Garbage Collectors (for Diagnosis):
   
   set these values -Xms1024m -Xmx2048m -XX:+UseParallelGC
   
   in proxyserver.sh:
   

   JVM_PERF_OPTS="-server -XX:MaxMetaspaceSize=256M -XX:+UseParallelGC"
   JVM_MEM_OPTS="-Xms1024m -Xmx2048m"