DX NetOps Performance Management Threshold Monitoring Suspended

Products

• CA Infrastructure Management
• CA Performance Management
• Network Observability

Issue/Introduction

Dx NetOps Performance Management is not creating Threshold  Violation events

Event Processing dashboards suddenly show no events being processed

Environment

Any version of Dx NetOps Performance Management

Cause

The threshold limiter monitors how long the evaluation engine takes to process rules in the Data Aggregator. If the threshold monitoring exceeds the specified percentage of the poll cycle, the evaluation engine enters a DEGRADED state. In the DEGRADED state, the evaluation engine waits for the monitoring to drop below the specified percentage. After a specified time, the threshold monitoring engine reassesses whether to suspend threshold evaluations. If threshold violations continue to exceed the percentage during the time period, the evaluation engine is suspended. Threshold evaluations will not resume, even if you restart the Data Aggregator. If the threshold violation does not exceed the specified percentage, the evaluation engine returns to normal operation.

Important! Do not modify the threshold limiter settings. The default settings provide protection against potential polled data loss. For any changes to the limiter settings, contact Broadcom Support.

Resolution

Follow these steps:

1. Enter the following information into a web browser:

   • http(s)://DA_host:port/rest/thresholdmonitoring/config

2. Take note of the ID value of the ThresholdMonitoringConfiguration item. For example we see the ID is 16 in this output.

   <ThresholdMonitoringConfigurationList>
      <ThresholdMonitoringConfiguration version="1.0.0">
       <ID>16</ID>
       <ThresholdMonitoringEnabled>true</ThresholdMonitoringEnabled>
        <PercentOfPollCycleThreshold>80</PercentOfPollCycleThreshold>
        <ThresholdMonitoringLimiterEnabled>true</ThresholdMonitoringLimiterEnabled>
        <RecoveryIntervalInMinutes>15</RecoveryIntervalInMinutes>
      </ThresholdMonitoringConfiguration>
   </ThresholdMonitoringConfigurationList>

3. Open a REST client editor or HTTP tool that sends requests and gets responses.
4. Set the Content-type to application/xml.
5. Enter the following filter criteria:
   
   • URL: http://DA_host:port/rest/thresholdmonitoring/config/ID
     
     • ID: The identification number that is assigned to the ThresholdMonitoringConfiguration item.
   • HTTP method = PUT
   • BODY content: True enables, False disables to shut down event evaluations to halt event raises.

     • <ThresholdMonitoringConfiguration version="1.0.0">
            <ThresholdMonitoringEnabled>true</ThresholdMonitoringEnabled>
       </ThresholdMonitoringConfiguration>

Can't use a REST API UI? Use the following curl commands.

1. Get the ID using:
   • curl -kv -u <user> http(s)://<da_hostname>:<PORT>/rest/thresholdmonitoring/config/
     • Replace <user> with a user with administrator role access to Portal
     • Set the correct DA scheme (https vs https), hostname and port.
     • Enter the user password when prompted
   • Note the <ID> value returned for the next step.
2. Run the following curl. Set to true to enable and false to disable event raises.
   • curl -kv -u <user> -X PUT -H "Content-Type: application/xml" -d '<ThresholdMonitoringConfiguration version="1.0.0"><ThresholdMonitoringEnabled>true</ThresholdMonitoringEnabled></ThresholdMonitoringConfiguration>' http(s)://<da_hostname>:<PORT>/rest/thresholdmonitoring/config/<ID>
     • Replace <user> with a user with administrator role access to Portal
     • Set ThresholdMonitoringEnabled to true to enable, false to disable
     • Set the correct DA scheme (https vs https), hostname and port.
     • Replace <ID> with the ID returned with the current config settings.
     • Enter the user password when prompted
3. Run the first get again to validate the value change.

Additional Information

Threshold Monitoring and Threshold Limiter Behavior