APM 10.7 Collectors OutOfMemory due to the "Differential Analysis" configuration

Products

• CA Application Performance Management (APM / Wily / Introscope)

Issue/Introduction

Some collectors are running out of memory

heapdumps have been generated and Heap Analyzer indicates that the problem could be related to the Baseline engine, below an example:

 

Environment

Release : 10.7.0

Cause

The heap dump shows the execution queue for the baseline metric calculating thread has grown too big.

The Collector log has error indicating possibly corrupted baseline db:

1/19/23 03:15:53 AM CET [ERROR] [com.ca.apm.baseline.em.Baseline] 

org.garret.perst.StorageError: Object access violation: java.io.StreamCorruptedException: invalid type code: 50
    at org.garret.perst.impl.StorageImpl.loadStub(StorageImpl.java:2831)
...

Resolution

Recommendations:

1) Remove baseline db from Collectors

a) stop collector(s)

b) remove the baseline database, it is defined in the IntroscopeEnterpriseManager.properties, property introscope.enterprisemanager.baseline.database

c) start the collector

 

2) Disable any custom Differential Controls from Webview > Management

For more information about Differential Analysis feature, see 

Create a Differential Control

 

Additional Information

APM 10.x - Troubleshooting and Best Practices