search cancel

DX APM java agent consuming lot of heap/cpu and lead to lot of GC activity due to Async call endless loop

book

Article ID: 216215

calendar_today

Updated On:

Products

CA Application Performance Management (APM / Wily / Introscope)

Issue/Introduction

Symptoms:
- Slow response times of application and/or application is not responding
- High cpu and heap utilization
- GC behavior is abnormal (a lot of GC Invocations Per Interval Count in G1 Young Generation)
- Even with non or low traffic on application, the issue occurs shortly after re-start of application
 

Environment

APM Java Agent 10.7x and 2x

Cause

Related to defect DE502669 and DE504059: APM Agent - Async call endless loop

Heapdump Analysis:

"there are two RxComputationScheduler threads that keep running FutureTasks that create and schedule new FutureTasks causing an endless chain of tasks. Somehow one of these tasks has been associated with a frontend so all these FutureTask are tracked as part of this transaction.
It seems they are using RxJava with servlets to access postgres database through hibernate. We have not tested RxJava with our async tracers"

 

Resolution

To be fixed in in APM Java SaaS version 21.9

 

Current workaround:

Turn off this property in extensions/java-async/bundle.profile

introscope.autoprobe.enable.tracergroup.AsyncFutureTracing=false

You can pass this property as a Java argument using -D parameter (as -Dintroscope.autoprobe.enable.tracergroup.AsyncFutureTracing=false)

 

Additional Information

https://techdocs.broadcom.com/us/en/ca-enterprise-software/it-operations-management/dx-apm-saas/SaaS/DX-APM-Saas-Whats-New.html

APM Introscope .Java Agent - Troubleshooting and Best Practices
https://knowledge.broadcom.com/external/article/145354/apm-introscope-java-agent-troubleshooti.html