Resume threshold monitoring evaluations from the command line

Products

CA Performance Management - Usage and Administration DX NetOps

Issue/Introduction

How can I resume threshold monitoring evaluations from the command line?

Threshold Monitoring Disabled
Threshold Limiter Enabled

Data Aggregator in DX NetOps Portal shows Threshold Evaluations are Shutdown in System Status.

No longer see new Threshold Violation Events in DX NetOps Portal

Environment

All supported DX NetOps Performance Management releases

Cause

The Threshold Limiter was tripped, halting further threshold evaluations, to protect the Data Aggregator from going down. Normally due to low resources, or too much threshold evaluation processing overcoming the Data Aggregator resources due to too many or misconfigured threshold rules.

Resolution

Follow these instructions to resume threshold evaluations using a curl-based REST call on the DA or Portal terminal CLI. This walks through how to update the ThresholdMonitoringEnabled value from false to true. This will cause threshold evaluations to resume.

Where applicable replace sample values like these.

<scheme> with HTTP or https
<DA_Host> with the DA hostname
<Port> with the DA port
<user> with a valid Portal UI administrative user

Determine the ThresholdMonitoringConfiguration <ID> value.

Option A: Use a browser.
1. Enter the following URL into a web browser:
  - <scheme>://<DA_Host>:<Port>/rest/thresholdmonitoring/config
2. Log in with a valid Portal UI administrative user
3. Note the <ID> value for the ThresholdMonitoringConfiguration.
4. Sample output from a support lab:
Option B: Use a curl REST call on the DA or Portal terminal CLI.
1. Run the following:
  - curl -kv -u <user> <scheme>://<DA_Host>:<Port>/rest/thresholdmonitoring/config
2. Enter the user password when prompted.
3. Look for a response of 200 OK like this. Anything else is an error.
  - < HTTP/1.1 200 OK
4. The data at the end after a 200 OK should look like this sample from a support lab. Note the highlighted <ID> value.
  - * Connection #0 to host 10.253.128.39 left intact
    <ThresholdMonitoringConfigurationList><ThresholdMonitoringConfiguration version="1.0.0"><ID>4942</ID><ThresholdMonitoringEnabled>true</ThresholdMonitoringEnabled><RecoveryIntervalInMinutes>15</RecoveryIntervalInMinutes><ThresholdMonitoringLimiterEnabled>true</ThresholdMonitoringLimiterEnabled><LinkedRulesProcessingEnabled>true</LinkedRulesProcessingEnabled><PercentOfPollCycleThreshold>80</PercentOfPollCycleThreshold></ThresholdMonitoringConfiguration></ThresholdMonitoringConfigurationList>

Change the ThresholdMonitoringEnabled value from false to true.

Run the following command from the DA or Portal terminal CLI:
- curl -kv -u <user> -H "Content-Type:application/xml" -X PUT <scheme>://<DA_Host>:<Port>/rest/thresholdmonitoring/config/<ID> --data "<ThresholdMonitoringConfiguration version='1.0.0'><ThresholdMonitoringEnabled>true</ThresholdMonitoringEnabled></ThresholdMonitoringConfiguration>"
Look for a response of 200 OK like this. Anything else is an error.
- < HTTP/1.1 200 OK
If a 200 OK is observed, refresh the /rest/thresholdmonitoring/config data using the same steps used to find the <ID>. The ThresholdMonitoringEnabled should now be true.

The Portal System Status for the DA should also be updated to reflect the change. At this point, you should begin to see Threshold Violation Events again when threshold violations are observed.

Additional Information

The Threshold Monitoring and Threshold Limiter Behavior documentation topic shows:

The REST API method to change ThresholdMonitoringEnabled from false to true in the Resume Threshold Evaluations section.
- It has the same steps as below for obtaining the ThresholdMonitoringConfiguration ID.
- It has the BODY example required to update ThresholdMonitoringEnabled from false to true.
Tips about where to look and what Dashboards to review to examine the Thresholding state and what may have caused it to be disabled.