JMX Metric not reporting real value

Products

• CA Application Performance Management Agent (APM / Wily / Introscope)

Issue/Introduction

 

 

Introscope was not reporting a good value for the JMX metric "Consumers Current Count", the specific metric is the following:

• *SuperDomain*|myhost|Weblogic|WeblogicAgent|JMX|com.bea|JMSServerRuntime=jmshost|Name=module_jms!jmsserver@app|ServerRuntime=jmshost|Type=JMSDestinationRuntime:ConsumersCurrentCount

Right after a restart of the JMS instances, the metric went down from 288 (normal value) to 14 which caused a configured Introscope alert to be raised.

After checking in Weblogic itself, it was reporting a value of 288, contrary to the 14 that Introscope reported.

What could be the cause for this ?

 

Environment

Release : 10.7.0

Component : APM Agents

Cause

 

 

Enabling agent DEBUG logging gives this error:

[DEBUG] [IntroscopeAgent.Agent] Another instance of IntroscopeStartupClass is already running

 

There are a few ways to invoke JMX monitoring and start the JMX service in the agent:

1) Prior to APM 10.7, an Introscope Startup Class would be configured in Weblogic to enable JMX data to be reported

2) From APM 10.7 onwards, a startup tracer has been included which allows JMX data to be collected without extra configuration in Weblogic

3) There is another option to be able to enable JMX monitoring through an API.

 

In this case, it appears that both the startup tracer and API service ( 2 & 3) were running at the same time causing a conflict in the data reported by Introscope.

Resolution

 

 

 

The API that could be used to trigger starting JMX service has been disabled as there is no valid use case for it, the service will be started through the startup tracer.

A hotfix version of the agent will include this change.

Additional Information

 

It is possible to get a similar problem by enabling the options 1) and 2) which is unnecessary, so it should be verified that the IntroscopeStartupClass is not configured in Weblogic.