Agent shut off suddenly in investigator

Products

• CA Application Performance Management (APM / Wily / Introscope)

Issue/Introduction

We are experiencing strange behavior with some agents. These agents suddenly stay in a "shut off" state in the investigator, and when we realize it, we have to remove the "shut off" check from investigator so the agent reports metrics again.

In the log files of our collector we only see that it disconnects from one collector and connects to the other, but in investigator view we see that once it disconnects from the collector, it does not report metrics. We discard a network problem because in the same host we have more agents that reports metrics without problems.

 

Environment

Release : 10.8.0

Cause

 The configuration for the shutoff is held in a file  {EM_HOME}/config/shutoff/MetricShutoffConfiguration.xml.

 

This file is held on each collector and when updated for an agent it only updates it in the collector that the agent is currently connected to. So if an agent is connected to collector 1 and then for some reason is reassigned to collector 2 BUT on collector 2 the agent had previously been shutoff then it will inherit that condition.

Resolution

One collector contained multiple shutoff records - these were cleared as follows

• Stop the Enterprise Manager
• Delete the file {EM_HOME}/config/shutoff/MetricShutoffConfiguration.xml.
• Restart the Enterprise Manager