SNMPv3 IP discovery fails using both seedfile and using GUI, but SNMPv3 sm_snmpwalk using same credentials is successful.

Products

• VMware Smart Assurance
• VMware Telco Cloud Service Assurance

Issue/Introduction

SNMPv3 IP discovery fails using both seedfile and the Smarts IP GUI, but SNMPv3 sm_snmpwalk using same credentials is successful
Device added to Smarts IP pending list with the following message:

No response from SNMP agent, V3, agent SNMP-E-EAGENTUSMUSER-Agent
REPORT [USM]: Unknown User Name '<userName>'

Following messages seen in Smarts IP log file:

ADTM-N-AD_LOG-23-Jun-2009 10:37:16 AM, at 'Callback, bad response', while
    discovering '192.168.111.5',

  . SNMPError = SNMP-E-ERESPONSE-No response from 192.168.111.5, port 161
  .  SNMP-ETIMEOUT-Timed out

Environment

Smarts - 10.1.x

TCSA - 2.x

Cause

Multiple SNMP agents are running the same EngineID. If more than one device is using the same EngineID, there will be collisions in the LCD (Local Configuration Datastore) table and will get the "Unknown User" messages described above. This scenario is unsupported because of the way the RFCs define how the LCD  is to be indexed (by engine id and user id). This tuple is the unique key pair into the credentials table where passwords are stored. 

This is important when either authentication or privacy are used for the v3 devices. In essence, you are  "spoofing" one agent with another but duplicating its engine ID and credentials. The engine IDs need to be unique as they are used as part of the encryption key. Once this uniqueness is broken, you cannot guarantee that an authenticated packet came from where it claims to have come from.

Resolution

1. Check and ensure that there are no SNMP agents running the same EngineID. You can use packet traces and the output of the following commands to determine this:
   
   sh snmp engineID
   sh snmp user
   
   
   For example, two switches set up with exact same SNMP engineID and user would have output similar to the following:
   
   #sh snmp engineID
   Local SNMP engineID: ########################
   Remote Engine ID          IP-addr    Port
   
   #sh snmp user
   User name: <userName>
   Engine ID: ########################
   storage-type: nonvolatile        active
   Authentication Protocol: MD5
   Group-name: <groupName>
   
   
2. Reset the engineID and user. To resolve this issue using the preceding engineID/user example, the following commands would be used:
   
   #no snmp-server user <userName> <userGroup> v3 auth md5 changeme
   #no snmp-server <userName> <userGroup> v3 auth
   #snmp-server engineID local ########################
   #snmp-server <userName> <userGroup> auth
   #snmp-server user <userName> <userGroup> v3 auth md5 changeme
   
   Note: Replace <userName>, <userGroup>, and ######################## with the correct values.
   IMPORTANT! Be sure to reset both the engineID and user. Resetting just the engine ID will result in incorrect engineId/user bindings. 
   
   Once the engineID and user have been reset, the SNMPv3 IP discovery should succeed.

P.S: Correcting UserName and EngineID is responsibility of device owner. This article can be used for reference however support will have limited knowledge and access.