Unable to discover SNMPv3 based device in CA Performance Management (CAPM)

Products

• CA Infrastructure Management
• CA Performance Management - Usage and Administration
• DX NetOps

Issue/Introduction

Updating devices to utilize SNMPv3 instead of SNMPv2c.

After changing the devices SNMP Profile to the correct SNMPv3 based profile, the device switches back to its old still valid SNMPv2c profile.

If the old SNMPv2c Profile is deleted, and the device is set to the correct SNMPv3 based profile, it first shows Green/Normal for Status in the Data Aggregator Inventory. After one poll cycle it changes to Contact Lost for the status.

If deleting the device and trying to rediscover it as new, it is only found as a Pingable.

This is all despite confirming that the correct credentials return a valid response using SNMP request tools at the command line of the Data Collector that will manage and poll these devices.

The IP for the problem device is found in the following ERROR message in the managing Data Collectors karaf.log file. It will list each IP showing the same EngineID in use for an SNMPv3 device. In this message example there are four devices using the same EngineID.

2020-02-12 12:43:52,863 | ERROR | Response Pool.2  | MPv3Proxy                        | m.ca.im.dm.snmp.snmp4j.MPv3Proxy  978 | 196 - com.ca.im.data-collection-manager.snmp - 3.7.1.RELEASE-384 |  | Duplicate EngineID detected: 80:00:00:09:03:00:00:00:00:00:00:00, IP(s) associated [<IP_Addr_1>/161, <IP_Addr_2>/161, <IP_Addr_3>/161, <IP_Addr_4>/161]

Default path to the Data Collector karaf.log is:

/opt/IMDataCollector/apache-<version>/data/log

Environment

All supported Performance Management releases

Cause

Devices are configured in a way that duplicates the engine ID between devices.

Every SNMPv3 device must have a unique engine ID not shared by other devices.

This is described in RFC 5343 found here: https://tools.ietf.org/html/rfc5343

To validate is this is the cause we can use information from the DC snmpSession data. To access it we'll use different methods depending on the Performance Management release involved.

1. For all r20.2.x and earlier releases follow these steps.
   1. Open a new browser tab.
   2. Go to the following URL targeting the Data Collector (DC) host that will discover, poll and manage the device(s).
      1. http://<DC_HOST>:8681/debug/dispatcher/container/com.ca.im.data-collection-manager.snmp/bean/snmpSession
      2. Replace <DC_HOST> with the hostname or IP address of the target Data Collector host.
2. For all r21.2.1 and newer releases follow these steps.
   1. Due to new security enhancements the URL is only functional via curl command on the DC terminal CLI using the loopback address.
   2. Open a terminal to the DC and issue the following command as the root or install owner user:
      1. curl http://127.0.0.1:8681/debug/dispatcher/container/com.ca.im.data-collection-manager.snmp/bean/snmpSession

In the output from that URL search for the entries that show known SNMPv3 engine ID values and IP Addresses that have reported them. Where there is more than one IP Address against a given SNMPv3 engine ID, we have multiple devices reporting the same engine ID.

Sample section showing three devices reporting the same engine ID.

MPv3 engineIDs: 

81:43:45:0f:87:32:7b:23:c6:5c:4b:2e:66 : 

EngineBoots: 8, EngineTime: 2999050 

MULTIPLE ADDRESSES:[1.1.1.1/161, 2.2.2.2/161, 3.3.3.3/161] 

Note that the cache is not cleared until the DC is restarted so if the engineID has changed on the device you may see still see the older entry.

Resolution

Reconfigure affected devices to ensure they have unique SNMPv3 engine ID values.