JVM crashed with APM Java agent enabled

Products

• DX APM SaaS

Issue/Introduction

Have seen the issue where couple of times Java applications crashing on two systems. Both Java applications have APM agent enabled. We are suspecting the APM Java agent is causing these crashes.

Cause

Crash logs showed the last problem frame in VM code upon some GC operation.

A rare bug in the JVM. We see a large number of Java threads created and all of them in a blocked state, so it looks like a GC thread is encountering a bug.

Review of the wily directory seems to confirm that environment is using 2023.10 agent with all the default agent extension bundles being deployed and without any obvious errors logged that could be correlated to the reported JVM crashes.

No native component (platform monitoring) explicitly bundled with agent. 

Resolution

These crashing are not occurring due to APM agent.

There is a Redhat KB found online discussing a similar issue:
"JVM crashes with a SIGSEGV at Par_MarkFromRootsClosure::scan_oops_in_oop(HeapWord*)" - https://access.redhat.com/solutions/887603

with apparently an older JRE version (1.6 vs in this case environment 1.7) with the following resolution:

Resolution
Upgrade to the latest 1.6 update as the used JRE version: 6.0_35-b10 is old and would be missing many fixes or

Upgrade to the latest 1.7 update.

Try to switch the GC algorithm.

Reproduce it on OpenJDK, and if there is an issue Red Hat can fix it .

Get in touch with Oracle or Engage Oracle to see if they have any JDK layer resolution for these crashes.

Please contact JVM vendor to proceed further on this issue and get a resolution on this issue from JVM vendor.