What devices are being monitored by a Data Collector?

Products

• Network Observability
• CA Performance Management

Issue/Introduction

Scenario A:

A Data Collector (DC) host is being decommissioned. The server where the DC was installed was removed and deprecated or deleted and is no longer available. This was done before ensuring all managed devices associated to the DC were moved to other DCs or deleted. The DC shows in the Portal UI as a greyed out DC entry that is not selectable but does show more than zero polled devices. How can we identify the devices that remain associated to the DC in order to remove them?

 

Scenario B:

Unable to Discover a Device in DX NetOps PM After the Data Collector to Which the Device Was Assigned Was Deleted Through Data Aggregator REST API. We are encountering an issue where a device cannot be discovered or monitored in DX NetOps Performance Management (PM) because its assigned Data Collector was deleted using the REST API.  As a result, the device remains in an undefined state, preventing proper discovery and monitoring.

Scenario:

Suppose device X is being monitored by Data Collector Y. Due to infrastructure changes, the network administrator decides to delete Data Collector Y via the Data Aggregator REST API:

http://DA:8581/rest/dcms/<ItemID of DC>

As a result, device X moves into an undefined state because its associated Data Collector no longer exists. Within the Data Aggregator device admin page, the administrator is unable to assign a new Data Collector to rediscover the device because the dropdown menu is greyed out.

Environment

DX NetOps Performance Management All releases

Cause

The issue arises when one of these scenarios are taking place.

• A device in DX NetOps Performance Management (PM) becomes associated with a Data Collector that was deleted using the REST API.
• A DCs host is removed before properly uninstalling the DC.

Resolution

To determine the devices that remain on a greyed out "ghost" DC follow these steps. Once the devices are known we delete them and rediscover them on existing DCs. We can also move them from one DC to another.

 

1. Go to the following URL against the Data Aggregator to determine the DCs ID. Replace <scheme>://<DAHost>:<Port> with the real DA details.
   •  <scheme>://<DAHost>:<Port>/rest/dcms
   • Default DA sample for DA host named DataAggregator: 
     • http://DataAggregator:8581/rest/dcms
   • Default DA sample for DA host named DataAggregator using HTTPS and default port: 
     • https://DataAggregator:8582/rest/dcms
2. Use the following curl command to generate a list of devices associated to a given DC.
   1. curl -k -u <userName> -H "Content-Type: application/xml" -X POST <scheme>://<DAHost>:<Port>/rest/devices/filtered -d '<FilterSelect xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="filter.xsd"><Filter><And><Item.MDRItemID type="EQUAL">DC_ID</Item.MDRItemID></And></Filter><Select use="exclude" isa="exclude"><Item use="exclude"><Name use="include"/></Item><Device use="exclude"><PrimaryIPAddress use="include"/></Device></Select></FilterSelect>'
   2. Replace the bold texts fields with legitimate values.
      • Replace <userName> with the a valid user with Portal access
      • Replace <scheme>://<DAHost>:<Port> with the real DA details.
      • Replace DC_ID with the DC ID identified in step 1.
   3. Add ">> /tmp/devicesOnDC.out" to the end of the curl command to generate an output file.

 

Once the devices are identified they need to be removed from association to the old DC.

 

1. Delete, or Delete and Rediscover.
   1. Go to the DA Inventory in the Portal UI. Find and select the target devices.
   2. Take action:
      1. Don't care about the devices current historical data, but need to continue polling them? Delete them and then rediscover them against an active remaining DC.
      2. Don't care about the devices and no longer need to monitor them? Just delete them.
2. Move the devices to another existing DC to resume polling.
   1. Go to the DA at /rest/dcms as done earlier in this KB. Find and note the MDRItemID of the new Data Collector that will monitor the device(s).
   2. Generate a text file with the device Item_IDs listed.
      
      1. NOTE: Ensure there is only one device Item_ID per line in the file.
   3. The following script could be used to iterate through a list of device IDs to change them.

      1. for itemid in `cat /tmp/devices.txt`
         do
         curl -X PUT -u <USER>:<PASSWORD> -H 'Content-Type: application/xml' -d'<Device version="1.0.0"><Item version="1.0.0"><MDRItemID>NEW_COLLECTOR_ID</MDRItemID></Item></Device>' <SCHEME>://<DA_NAME>:<PORT>/rest/devices/$itemid
         done

      2. Replace:
         1. <USER> and <PASSWORD> with those representing a valid Portal user.
         2. <SCHEMA>, <DA_NAME> and <PORT> to the values the DA utilizes.
         3. NEW_COLLECTOR_ID with the appropriate MDRItemID value set on the target active DC.
   4. After the process is completed the devices should show the new DC assigned in their Details tab in the DA Inventory views.

Additional Information

Note: Deleting a Data Collector via the REST API, or removing it's host from the environment without proper uninstall steps being followed, can leave devices in an undefined state. It is not recommended to delete a Data Collector if it is currently monitoring devices. Move existing devices to another DC or delete them prior to taking these steps on an active DC.