DX NetOps Performance Management Threshold Monitoring Suspended

Products

• CA Infrastructure Management
• CA Performance Management
• Network Observability

Issue/Introduction

Dx NetOps Performance Management is not creating Threshold  Violation events
Event Processing dashboards suddenly show no events being processed

Environment

Any version of Dx NetOps Performance Management

Cause

The threshold limiter monitors how long the evaluation engine takes to process rules in the Data Aggregator. If the threshold monitoring exceeds the specified percentage of the poll cycle, the evaluation engine enters a DEGRADED state. In the DEGRADED state, the evaluation engine waits for the monitoring to drop below the specified percentage. After a specified time, the threshold monitoring engine reassesses whether to suspend threshold evaluations. If threshold violations continue to exceed the percentage during the time period, the evaluation engine is suspended. Threshold evaluations will not resume, even if you restart the Data Aggregator. If the threshold violation does not exceed the specified percentage, the evaluation engine returns to normal operation.

Important! Do not modify the threshold limiter settings. The default settings provide protection against potential polled data loss. For any changes to the limiter settings, contact Broadcom Support.

Resolution

Follow these steps:

1. Enter the following information into a web browser:

    http(s)://DA_host:port/rest/thresholdmonitoring/config

2. Take note of the ID value of the ThresholdMonitoringConfiguration item.
   Example:

   
   

   <ThresholdMonitoringConfigurationList>
      <ThresholdMonitoringConfiguration version="1.0.0">
       <ID>16</ID>
       <ThresholdMonitoringEnabled>true</ThresholdMonitoringEnabled>
        <PercentOfPollCycleThreshold>80</PercentOfPollCycleThreshold>
        <ThresholdMonitoringLimiterEnabled>true</ThresholdMonitoringLimiterEnabled>
        <RecoveryIntervalInMinutes>15</RecoveryIntervalInMinutes>
      </ThresholdMonitoringConfiguration>
   </ThresholdMonitoringConfigurationList>

3. Open a REST client editor or HTTP tool that sends requests and gets responses.
4. Set the Content-type to application/xml.
5. Enter the following filter criteria:
   
   • URL: http://DA_host:port/rest/thresholdmonitoring/config/ID
     
     • ID
       The identification number that is assigned to the ThresholdMonitoringConfiguration item.
   • HTTP method = PUT

Resume threshold evaluations on the Body tab of the HTTP Request pane:

<ThresholdMonitoringConfiguration version="1.0.0">
     <ThresholdMonitoringEnabled>true</ThresholdMonitoringEnabled>
</ThresholdMonitoringConfiguration>

The curl command can be used from the command-line as follows; 

curl kv -X PUT -H "Content-Type: application/xml" -d '<ThresholdMonitoringConfiguration version="1.0.0"><ThresholdMonitoringEnabled>true</ThresholdMonitoringEnabled></ThresholdMonitoringConfiguration>' http(s)://<da_hostname>:<PORT>/rest/thresholdmonitoring/config/<ID>

Replace <ID>, <PORT>, <da_hostname>, and choose the protocol (http/https) accordingly)

 

Additional Information

Threshold Monitoring and Threshold Limiter Behavior