Agent Clamping - New metrics will not be accepted

Products

• CA Application Performance Management Agent (APM / Wily / Introscope)
• CA Application Performance Management (APM / Wily / Introscope)
• INTROSCOPE
• DX Application Performance Management

Issue/Introduction

From the Collector Log:

[Manager.Agent] The Agent BCTest1 is exceeding the per-agent metric clamp (current=501462, max=500000). New metrics will not be accepted

Environment

Release : All supported releases.

Component : APM Agents

Cause

Per-agent metric clamp is based on live and historical

Resolution

To address Agent metric clamp, see below :

1) Use SmartstorTool to reduce the current registered metrics in the Smartstor metadata Please refer to

https://knowledge.broadcom.com/external/article?articleId=93176 

Section B item 17.  

https://comm.support.ca.com/kb/_introscope-enterprise-manager-troubleshooting/KB000093176

Refer to section B, item#9 

2) Reduce or disable the unwanted metrics in the Agent profile, pbl or pbd as this is the source of the problem We normally recommend customers to temporally disable the below feature to reduce the overhead or huge amount of metrics However,this of course will depend on your requirements, 

a) disabling automatic entry point detection feature
introscope.agent.deep.entrypoint.enabled=false 
b) disable SmartInstrumentation:
introscope.agent.deep.instrumentation.enabled=false introscope.agent.deep.trace.enabled=false
introscope.agent.deep.errorsnapshot.enable=false
introscope.agent.deep.automatic.trace.crossprocess.enabled=false 
c) Disable JMX collection: set
introscope.agent.jmx.enable=false or reduce the jmx filter to reduce the # of metrics being gathered. 
d) Switch from full to typical pbl. The full profile provies greater visibility, but incurs higher overhead. It is recommended as a starting point for qa, staging and development environments. Full monitoring monitors what the typibcalPBL file monitors and more, include: sockets I/O, file system I/O, JNDI, EJB method level, JTA, XSLT, Java API for XML messaging(JAXM) 
e) Turn off the socket tracers in toggles typical/full pbd file. 
TurnOn:ManagedSocketTracing 
TurnOn: SocketTracing 
TurnOn: NIOSocketTracing 
TurnOn: NIOSocketSummaryTracing 
TurnOn: NIOSelectorTracing 
TurnOn: NIODatagramTracing 
TurnOn: NIODatagramSummaryTracing 

f) If the found that the isuse is cuased by the high # of SQL metrics, you can try
to disable/ reduce SQL instrumentation as below - Reduce the length of SQL statements. The default maximum length captured by the agent is 999. You can modify this by adding the following line to the
IntroscopeAgent.profile file: introscope.agent.sqlagent.sql.maxlength= - reduce sql gathering by using the following properties: 

introscope.agent.sqlagent.sql.turnoffmetrics=true
introscope.agent.sqlagent.sql.artonly=true 
introscope.agent.sqlagent.sql.turnofftrace=true 

- use New SQLAgent normalization mechanism 
- see Agent documentation. 
- Follow recommendations in 

https://techdocs.broadcom.com/us/en/ca-enterprise-software/it-operations-management/dx-apm-agents/24-5-1/java-agent/configure-java-monitoring/configure-sql-monitoring-for-java.html

Please refer under, ConfigureSQLMonitoringforJava- TurnOffSQLStatementMetrics

“You can turn off SQL statement metric reporting. You might do so when an application generates large numbers of unique SQL statements, causing a metric explosion. When you turn off
SQL statement metric reporting, the backend or top-level JDBC metrics are not lost...."