Azure Pipeline runs not showing metric needed

Products

• CA Application Performance Management (APM / Wily / Introscope)

Issue/Introduction

My Azure pipeline is not showing the needed metric. We are seeing other metrics but the most important one is not showing. I need help as soon as possible

Resolution

Not following the recommended heap settings caused this issue.

-Your APMIA wrapper logs showing running out of memory

INFO   | jvm 12   | java.lang.OutOfMemoryError: GC overhead limit exceeded
STATUS | wrapper  | The JVM has run out of memory.  Restarting JVM.

-Then it created the heap dump.

INFO   | jvm 12   | Dumping heap to logs/java_pid69652.hprof ...
INFO   | jvm 12   | Heap dump file created [518425929 bytes in 2.437 secs]

-Notice you are getting this in passing 

 SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".

This appears to be using a 99 agent 

[INFO] [InfrastructureAgent] CA APM Infrastructure Agent Release 99.99.0.nextgen_aws_restmon (Build 1272)

A 99 agent is usually a development build.

-Until you increased the heap, the Agent logs were filled with 

 ERROR] [IntroscopeAgent.RESTMon] [profile-9@@azure.3~~azure_datafactory ApmNodeProcessor] Error in getExternalIds for :%layer::AzurePipelineRun%//%:%//%$['value'][*]['pipelineRunId']%//%:%//%%pipelinename:%datafactoryname:%resgrp:%azure_subscriptionid
[ERROR] [IntroscopeAgent.RESTMon] [profile-9@@azure.3~~azure_datafactory ApmNodeProcessor] Error in getExternalIds for :%layer::AzureActivityRun%//%:%//%$['value'][*]['activity Name']%//%:%//%AzurePipelineRun%//%:%//%$['value'][*]['pipelineRunId']%//%:%//%%pipelinename:%datafactoryname:%resgrp:%azure_subscriptionid

So it clearly could not process the Azure metrics. 

-The Autoprobe log only showed 

==============
> #######################
> # Agent Specification
> # ===================
> # This is the main agent implementation class which handles
> # the logging and tracing of instrumentation data.
> 
> InstrumentTraceClass: com.wily.introscope.agent.AgentShim
> 
Finished file dump for /com/wily/introscope/probebuilder/pbd/required.pbd

Which may be due to the 99 agent. (Autoprobe heading) 

Introscope AutoProbe Log: /opt/CA/apmia/logs/AutoProbe.log
Release 99.99.0.nextgen_aws_restmon (Build 1272)(Diagnos Version 2)

In the hprof java heap dump. Top classes/memory

The class "com.ca.ce.restmon.util.ParsedContext", loaded by "com.wily.util.extension.JarExtension$AllPermissionsClassLoaderChildFirst @ 0xe01d1588", occupies 136,460,440 (32.95%) bytes. The memory is accumulated in one instance of "java.util.HashMap$Node[]" loaded by "<system class loader>".

The classloader/component "com.wily.util.extension.JarExtension$AllPermissionsClassLoaderChildFirst @ 0xe008b440" occupies 104,042,192 (25.02%) bytes. The memory is accumulated in one instance of "java.lang.Object[]" loaded by "<system class loader>".

8 instances of "io.netty.buffer.PoolChunk", loaded by "com.wily.util.extension.JarExtension$AllPermissionsClassLoaderChildFirst @ 0xe008b440" occupy 67,244,064 (16.17%) bytes. 

This is holding a lot of objects in memory when ran out of memory.

 

2) Sizing recommendations.

Did you see this in the documentation?

Download and Install the Infrastructure Agent on UNIX/Linux

The APM Infrastructure Agent has the following Memory Requirements:  The default minimum Java Heap Size is 256MB and the default maximum Java Heap Size is 512MB.

For Amazon Web Services (AWS), Microsoft Azure, and Google Cloud Platform (GCP), the recommended minimum and maximum heap size is 4GB.

You kept the minimum Java Heap Size at 246MB. You would change the value to 4 GB or 6 GB.