How to delete Retired Devices in Performance Management

Products

• CA Performance Management - Usage and Administration
• DX NetOps

Issue/Introduction

Once a Device is set to a Retired state, if the old historic data is unwanted, it is beneficial to system health to remove them from the system.

When also regularly running the remove_not_present_items.sh script to delete retired component items, first deleting retired devices may improve the time it takes for the  remove_not_present_items.sh script to complete. Deleting the device parent, deletes related items, and often will greatly reduce the retired item count pending deletion.

Provides a simple method to list the internal IDs representing Retired Devices to simply their removal via a deletion script.

Environment

All supported Dx NetOps Performance Management releases.

Resolution

The steps involved are:

1. Identify the Data Aggregator Data Source ID.
2. Create the list of IDs for the Retired Devices.
3. Run the script to delete the IDs in the resulting list.

To execute the steps complete the following:

1: Enter the CLI on the PC server and go to the (default path) /opt/CA/MySql/bin directory. Run the following command as the root, or sudo root user, that owns the installation:​​​​

./mysql -unetqos -pnetqos netqosportal -e "select sourceid from data_sources2 where sourcetype='262144';"

For example:

./mysql -unetqos -pnetqos netqosportal -e "select sourceid from data_sources2 where sourcetype='262144';"
mysql: [Warning] Using a password on the command line interface can be insecure.
+----------+
| sourceid |
+----------+
|        3 |
+----------+

2: Take the SourceID value from step 1 and run the following command. Change the directory, and filename the ID's are written to, as desired.

./mysql -unetqos -pnetqos netqosportal -e "select localid from dst_device where sourceid = '<ID>' and itemid in (select itemid from t_device where lifecyclestate='RETIRED');" > /tmp/retiredLocalIdsToDelete.out

 

HINT: To determine the number of ID values in the resulting file run "wc -l <filename>".

For example using the sample output from step 1 it might look like:
 

./mysql -unetqos -pnetqos netqosportal -e "select localid from dst_device where sourceid = '3' and itemid in (select itemid from t_device where lifecyclestate='RETIRED');" > /tmp/retiredLocalIdsToDelete.out

After create the retiredLocalIdsToDelete.out
Edit the File and remove the first line:
localid

Just need to have the numbers

3: Place the script attached to this article, deleteAllDevicesInFile.pl, on the PC server where the ID list file was created in step 2. Make the script executable (chmod +x <fileName>). Run it as follows:

 

deleteAllDevicesInFile.pl [-u user] [-pw password] [-da server] [-p httpsport] [-b batchsize] [-sl sleepSeconds] -f file

For example for DA named myda.company.com, using default non-https configs it might be:
 

deleteAllDevicesInFile.pl -u user -pw password -da myda.company.com -f /tmp/retiredLocalIdsToDelete.out

 

 

Using SSL enabled DA it might be (example port):

 

deleteAllDevicesInFile.pl -u user -pw password -da myda.company.com -p 8443 -f /tmp/retiredLocalIdsToDelete.out

 

If you got the message:
-bash: ./deleteAllDevicesInFile.pl: /usr/bin/perl^M: bad interpreter: No such file or directory

Just use the dos2unix to convert the file

By default the script will delete the items in batches of 500.

Additional Information

1. The script supports HTTPS enabled Data Aggregator servers running r3.6 or newer releases.
2. The script supports Data Aggregator authentication (user/password required for PM 21.2.x)
3. The deletions take place on the Data Repository via the Data Aggregator REST Services. Before we can expect to see results in PC Inventory a DA Update Sync cycle, followed by a subsequent PC Global Sync Cycle must be completed.
4. Run this before remove_not_present_items to lower the number of items it must work with and speed it's execution.
5. If the following error is returned by the deleteAllDevicesInFile.pl script a likely cause is a proxy in the environment.
   1. ERROR: unable to determine DA item_id
   2. Error is due to inability to get the DA ItemID from the following curl statement:
      1. curl -v -u admin -pw <password> -H 'Content-Type: application/xml' -X GET 'webScheme>://<DA_HOST>:<DA_Port>/rest/dataaggregator'
   3. Issue that same curl command at the CLI of the host where the error is seen replacing webScheme, <DA_HOST>, and <DA_Port>. It will return the error involved.
   4. If it shows a 403 error about a proxy host rejecting the service requested, the proxy needs to be removed from the equation.
   5. If this works, the --noproxy '*' part will need to be added to all curl commands in the script.
      1. curl --noproxy '*' -v -u admin -pw <password> -H 'Content-Type: application/xml' -X GET 'webScheme>://<DA_HOST>:<DA_Port>/rest/dataaggregator'