IP Discovery traffic caused a client impact

Products

• Network Observability
• Spectrum
• CA Performance Management

Issue/Introduction

An ip address found to exist in the discovery profile queue for the default domain spammed the client environment causing network connectivity issues for the client.  

The ip discoveries were returning "inaccessible" and the spamming was occurring from each of the data collectors in the ip domain.  

How can we prevent the spamming and only have the queued discovery profile to run once at a given time without repeating attempts?

 

We need a way to prevent this issue from recurring and determine if there is some monitoring we can apply to catch the problems sooner.

Environment

 DX NetOps :: All Supported Versions

Cause

When you integrate Spectrum with Performance Management in DX NetOps, it will create a Global Collection for each IPDomain in the Portal.

If you have chosen to 

“Contribute inventory to Data Aggregator”

on the Spectrum Data Source, then any device added to one of the IPDomain GC’s in spectrum will be sent to portal and added to the matching discovery profile (DefaultDomain by default).

This device will remain in that discovery profile until it is discovered as a manageable device in Portal, or is manually removed. 

As long as the device is in the profile, we will try to discover it every time the profile runs.

Resolution

To prevent devices from being discovered that are not wanted or not accessible, manage what devices are added to the GC’s on the spectrum side.

This can be done manually or using rules.

 

Additional Information

If you want to monitor when a device is failing discovery, you can parse this rest endpoint on the Data Aggregator (DA):

http:

http://DA:8581/rest/discoveryinstances

https:

     https://DA:8582/rest/discoveryinstances

You can scrape the results for:

<UnreachableDevicesList>

<UnreachableDevices>xx.xx.xx.xx</UnreachableDevices>

</UnreachableDevicesList>