Interfaces on few devices in Smarts have status NOTPRESENT when the status on MIB is UP
Article ID: 331771
Updated On:
Products
Issue/Introduction
Smarts is not polling interfaces properly. Many random interfaces are showing Admin or Oper status as NOTPRESENT, which is not the actual status on the MIB.
No changes have been done in device configuration or Smarts.
Environment
Smarts 10.X
Cause
1) Checked one interface as an example, it is instrumented for Fault by MIB2 in Smarts, with admin status UP and oper status NOTPRESENT.
-------------------------------------------
dmctl -s IPAM get Interface::IF-DSA/67149824::InstrumentedBy
{ Interface_Fault_MIB2::I-Interface_Fault_MIB2-IF-DSA/67149824 }
dmctl -s IPAM get Interface::IF-DSA/67149824::AdminStatus
UP
dmctl -s IPAM get Interface::IF-DSA/67149824::OperStatus
NOTPRESENT
-------------------------------------------
2) Checked the snmp output from device, both have value 1, which means they are UP:
-------------------------------------------
Admin Status:
sm_snmp -c public -d DSA get .1.3.6.1.2.1.2.2.1.7.67149824
.1.3.6.1.2.1.2.2.1.7.67149824 = 1
Oper Status:
sm_snmp -c public -d DSA get .1.3.6.1.2.1.2.2.1.8.67149824
.1.3.6.1.2.1.2.2.1.8.67149824 = 1
-------------------------------------------
3) Enabled trace on the device in Smarts
From the log, there is no evidence of polling details of Oper status of the interface, only Admin Status is captured.
-------------------------------------------
cat IPAM_en_US_UTF-8.log_trace | grep 67149824
73: OBJ-ID (0x06), 13 bytes == ".1.3.6.1.2.1.2.2.1.7.67149824"
76: OBJ-ID (0x06), 13 bytes == ".1.3.6.1.2.1.2.2.1.7.67149824"
73: OBJ-ID (0x06), 13 bytes == ".1.3.6.1.2.1.2.2.1.7.67149824"
76: OBJ-ID (0x06), 13 bytes == ".1.3.6.1.2.1.2.2.1.7.67149824"
-------------------------------------------
4) Captured debug data of SNMP Poller on the domain with below command:
dmctl -s <domain name> exec dmdebug --dumpAccessor=DEVSTAT-SNMP-Poller --output=accessor.log
From the log, there is an error on polling of Oper Status of the interface and the polling was last successful 1 day prior.
-------------------------------------------
Instance Name: I-Interface_Fault_MIB2-IF-DSA/67149824
Attribute Name: ifAdminStatus
Polling Period: 240
Last polled At: November 14, 2017 2:33:49 PM GMT+01:00
Cached Value: 1
Last Error: <OK>-E-<OK>-Normal successful completion
Unconstrained Polling Period: 15
Able To Poll: TRUE
-------------------------------------------
Instance Name: I-Interface_Fault_MIB2-IF-DSA/67149824
Attribute Name: ifOperStatus
Polling Period: 240
Last polled At: November 13, 2017 2:39:35 PM GMT+01:00
Cached Value:
Last Error: SNMP-E-ERR_NOSUCHNAME-No such name
Unconstrained Polling Period: 15
Able To Poll: FALSE
-------------------------------------------
5) The SNMP version in use for the device is V1:
-------------------------------------------
dmctl -s IPAM get SNMPAgent::SNMPAgent-DSA | grep SNMP
SNMPStatus = OK
SNMPVersion = V1
-------------------------------------------
For all the Non-V1 devices, Smarts never disables polling if it encounters any error. Which means for V2c or V3 discovered devices, if SMARTS gets any error while polling for an instance index, it just ignores them and continues to poll the index during next polling cycle.
But for V1 devices Smarts is default having the value (i.e. V1_continue_poll_after_error) set to FALSE. Which means that if there is any error while polling an instance index, Smarts will not poll that any more.
Resolution
Follow the below steps to set the same:
1) Check for the current value of the flag with below command:
dmctl -s <IP domain name> get ICIP_SNMPAccessorInterface::DEVSTAT-SNMP-Poller::V1_continue_poll_after_error
Ex:
dmctl -s IPAM get ICIP_SNMPAccessorInterface::DEVSTAT-SNMP-Poller::V1_continue_poll_after_error
FALSE2) Set the value to TRUE with below command:
dmctl -s <IP domain name> put ICIP_SNMPAccessorInterface::DEVSTAT-SNMP-Poller::V1_continue_poll_after_error TRUE
3) Execute the command of Step(1) to check for the value update.
4) Execute Reconfigure on the IP domain with below command:
dmctl -s <IP domain name> invoke ICF_PolicyManager::ICF-PolicyManager reconfigure
Check for Reconfigure status
5) Once reconfigure is complete, save the IP domain repository with below command:
sm_tpmgr -s <IP domain name> save
6) Rediscover the device (or) Execute "Discover All" on the domain.
Feedback
