How do you validate if the data presented by Performance Management matches what a device is reporting
All Supported Releases
Below is an example of validating the Discards out metric, it would be a similar procedure for any SNMP metric.
Acquire Preliminary Information:
1. Acquire the device's IP Address and the Interface in question's name via the Device/Interface page in Performance Center.
2. Access Administration > Monitored items Management > Monitored Devices > Lookup the device and in the details tab, take note of the IP Domain the device is a part of
3. In the Polled Metric Families tab of the same section, navigate down to find the Interface Metric family and confirm the certification is High Speed Interface or Interface.
4. For both of these certifications, the Discards Out metric data is provided by the IfOutDiscards object and it's OID is 1.3.6.1.2.1.2.2.1.19.index where the index is the index of the interface in question.
Using DCDEBUG to obtain raw polled values:
1. Access DCDEBUG in a new browser tab: http://DAhostname:8581/dcdebug
2. Enter the IP Address and IP Domain of the device in question, click the radio button next to Polling Configuration by IP then click "View Data" at the bottom
3. Navigate down until you find the line:
Normalized Facet Type={http://im.ca.com/normalizer}NormalizedPortInfo
4. Within this section, find your interface and record the Item ID and Index of it. For example, I am looking for Fa0/0.1 so will record Item ID 65434 and Index 8:
Polling Config Item ID=99212
Poll Group ID=5219
Normalized Facet Type={http://im.ca.com/normalizer}NormalizedPortInfo
Certification Facet Type={http://im.ca.com/certifications/snmp}IfTableMib
PollingConfig's DcmID=lvntest001021.bpc.broadcom.net:944bb919-8f69-4e35-945b-0a08a03f7f1b
Polled Items(8)
Item ID=65430, Name=Se0/0/0, Index List={[1]}
Item ID=65431, Name=Se0/1/0, Index List={[2]}
Item ID=65432, Name=Fa0/0, Index List={[3]}
Item ID=65433, Name=Fa0/1, Index List={[4]}
Item ID=65434, Name=Fa0/0.1, Index List={[8]}
Item ID=65435, Name=Fa0/0.729, Index List={[9]}
Item ID=65436, Name=Mu1, Index List={[10]}
Item ID=65437, Name=Se0/2/0, Index List={[11]}
5. Go back to the DCDEBUG main page, and click the Enable/Disable link to the right of Detailed Poll Logging by IP
6. Enter the IP Address & IP Domain if necessary and also, change Filtering to ItemIDs, and in the new ID List box, enter the Item ID from step 4. Click Enable Logging
7. Back on the DCDEBUG main page, enter the IP Address & IP Domain if necessary and click the radio button to the left of Detailed Poll Logging by IP and click the "View Data" button.
Analyzing the raw data:
1. Each poll will contain 3 entries, the list of OIDs to poll with all ending with NULL, the GET request of the OIDs, again all with Null, and lastly, the RESPONSE with the data values. For example, this is the RESPONSE for the example interface:
Jun 23 13:20:10.325: 1.3.6.1.2.1.2.2.1.19.8 = 115269860
Jun 23 13:20:10.325: 1.3.6.1.2.1.2.2.1.7.8 = 1
Jun 23 13:20:10.325: 1.3.6.1.2.1.2.2.1.14.8 = 115026806
Jun 23 13:20:10.325: 1.3.6.1.2.1.2.2.1.5.8 = 100000000
Jun 23 13:20:10.325: 1.3.6.1.2.1.2.2.1.20.8 = 114940949
Jun 23 13:20:10.325: 1.3.6.1.2.1.2.2.1.11.8 = 115340804
Jun 23 13:20:10.325: 1.3.6.1.2.1.2.2.1.15.8 = 115016249
Jun 23 13:20:10.325: 1.3.6.1.2.1.2.2.1.17.8 = 115217678
Jun 23 13:20:10.325: 1.3.6.1.2.1.2.2.1.8.8 = 1
Jun 23 13:20:10.325: 1.3.6.1.2.1.2.2.1.16.8 = 2764880860
Jun 23 13:20:10.325: 1.3.6.1.2.1.2.2.1.12.8 = 115237801
Jun 23 13:20:10.325: 1.3.6.1.2.1.2.2.1.18.8 = 115212000
Jun 23 13:20:10.325: 1.3.6.1.2.1.2.2.1.13.8 = 115251554
Jun 23 13:20:10.325: 1.3.6.1.2.1.2.2.1.10.8 = 2764880860
Jun 23 13:20:10.325: 1.3.6.1.2.1.1.3.0 = 20 days, 22:16:47.00
2. As the IfOutDiscards metric is a COUNTER we need to take the difference of two polls to get our singular data point. This is poll 2 that matches the above poll 1 from step 1 here:
Jun 23 13:25:10.327: 1.3.6.1.2.1.2.2.1.19.8 = 115290465
Jun 23 13:25:10.327: 1.3.6.1.2.1.2.2.1.7.8 = 1
Jun 23 13:25:10.327: 1.3.6.1.2.1.2.2.1.14.8 = 115046359
Jun 23 13:25:10.327: 1.3.6.1.2.1.2.2.1.5.8 = 100000000
Jun 23 13:25:10.327: 1.3.6.1.2.1.2.2.1.20.8 = 114960943
Jun 23 13:25:10.327: 1.3.6.1.2.1.2.2.1.11.8 = 115362141
Jun 23 13:25:10.327: 1.3.6.1.2.1.2.2.1.15.8 = 115034657
Jun 23 13:25:10.327: 1.3.6.1.2.1.2.2.1.17.8 = 115235682
Jun 23 13:25:10.327: 1.3.6.1.2.1.2.2.1.8.8 = 1
Jun 23 13:25:10.327: 1.3.6.1.2.1.2.2.1.16.8 = 3702380860
Jun 23 13:25:10.327: 1.3.6.1.2.1.2.2.1.12.8 = 115255456
Jun 23 13:25:10.327: 1.3.6.1.2.1.2.2.1.18.8 = 115231101
Jun 23 13:25:10.327: 1.3.6.1.2.1.2.2.1.13.8 = 115270203
Jun 23 13:25:10.327: 1.3.6.1.2.1.2.2.1.10.8 = 3702380860
Jun 23 13:25:10.327: 1.3.6.1.2.1.1.3.0 = 20 days, 22:21:47.00
3. Our two data points are:
Poll 1 = 1.3.6.1.2.1.2.2.1.19.8 = 115269860
Poll 2 = 1.3.6.1.2.1.2.2.1.19.8 = 115290465
4. The difference between then would be:
115290465
- 115269860
20605 Discards Out
5. If you scroll a bit further down in the DCDEBUG page you will find the summary of the poll (Poll Response after data processing section) that Performance Management will be importing into the database:
Jun 23 13:25:10.328: cycleTimestamp=1592933100000, readTimeStamp=0, duration=300003, pollRate=-1, deviceItemId=5912, dcmdItemId=0, rowData=[
Jun 23 13:25:10.328: DCMResponseVariable [name={http://im.ca.com/normalizer}NormalizedPortInfo.FrameSize, value=NULL ]
Jun 23 13:25:10.328: DCMResponseVariable [name={http://im.ca.com/normalizer}NormalizedPortInfo.ErrorsIn, value=19553 (Long) ]
Jun 23 13:25:10.328: DCMResponseVariable [name={http://im.ca.com/normalizer}NormalizedPortInfo.FramesIn, value=95602 (Long) ]
Jun 23 13:25:10.328: DCMResponseVariable [name={http://im.ca.com/normalizer}NormalizedPortInfo.PctDiscards, value=22.650110209686915 (Double) ]
Jun 23 13:25:10.328: DCMResponseVariable [name={http://im.ca.com/normalizer}NormalizedPortInfo.HistoricalInterfaceSpeed, value=100000000 (Long) ]
Jun 23 13:25:10.328: DCMResponseVariable [name={http://im.ca.com/normalizer}NormalizedPortInfo.SpeedIn, value=100000000 (Long) ]
Jun 23 13:25:10.328: DCMResponseVariable [name={http://im.ca.com/normalizer}NormalizedPortInfo.FrameSizeIn, value=NULL ]
Jun 23 13:25:10.328: DCMResponseVariable [name={http://im.ca.com/normalizer}NormalizedPortInfo.PctDiscardsIn, value=19.50691408129537 (Double) ]
Jun 23 13:25:10.328: DCMResponseVariable [name={http://im.ca.com/normalizer}NormalizedPortInfo.NonunicastIn, value=17655 (Long) ]
Jun 23 13:25:10.328: DCMResponseVariable [name={http://im.ca.com/normalizer}NormalizedPortInfo.Frames, value=173306 (Long) ]
Jun 23 13:25:10.328: DCMResponseVariable [name={http://im.ca.com/normalizer}NormalizedPortInfo.Availability, value=100 (Integer) ]
Jun 23 13:25:10.328: DCMResponseVariable [name={http://im.ca.com/normalizer}NormalizedPortInfo.Bits, value=NULL ]
Jun 23 13:25:10.328: DCMResponseVariable [name={http://im.ca.com/normalizer}NormalizedPortInfo.UtilizationIn, value=NULL ]
Jun 23 13:25:10.328: DCMResponseVariable [name={http://im.ca.com/normalizer}NormalizedPortInfo.Nonunicast, value=36756 (Long) ]
Jun 23 13:25:10.328: DCMResponseVariable [name={http://im.ca.com/normalizer}NormalizedPortInfo.PctErrors, value=22.81917533149458 (Double) ]
Jun 23 13:25:10.328: DCMResponseVariable [name={http://im.ca.com/normalizer}NormalizedPortInfo.BitsPerSecond, value=NULL ]
Jun 23 13:25:10.328: DCMResponseVariable [name={http://im.ca.com/normalizer}NormalizedPortInfo.Unicast, value=39341 (Long) ]
Jun 23 13:25:10.328: DCMResponseVariable [name={http://im.ca.com/normalizer}NormalizedPortInfo.BitsPerSecondOut, value=0.0 (Double) ]
Jun 23 13:25:10.328: DCMResponseVariable [name={http://im.ca.com/normalizer}NormalizedPortInfo.BitsOut, value=NULL ]
Jun 23 13:25:10.328: DCMResponseVariable [name={http://im.ca.com/normalizer}NormalizedPortInfo.ErrorsOut, value=19994 (Long) ]
Jun 23 13:25:10.328: DCMResponseVariable [name={http://im.ca.com/normalizer}NormalizedPortInfo.UnicastOut, value=18004 (Long) ]
Jun 23 13:25:10.328: DCMResponseVariable [name={http://im.ca.com/normalizer}NormalizedPortInfo.PctErrorsOut, value=25.730979100175023 (Double) ]
Jun 23 13:25:10.328: DCMResponseVariable [name={http://im.ca.com/normalizer}NormalizedPortInfo.BitsIn, value=NULL ]
Jun 23 13:25:10.328: DCMResponseVariable [name={http://im.ca.com/normalizer}NormalizedPortInfo.AdminStatusPollable, value=1 (Long) ]
Jun 23 13:25:10.328: DCMResponseVariable [name={http://im.ca.com/normalizer}NormalizedPortInfo.Errors, value=39547 (Long) ]
Jun 23 13:25:10.328: DCMResponseVariable [name={http://im.ca.com/normalizer}NormalizedPortInfo.Discards, value=39254 (Long) ]
Jun 23 13:25:10.328: DCMResponseVariable [name={http://im.ca.com/normalizer}NormalizedPortInfo.SpeedOut, value=100000000 (Long) ]
Jun 23 13:25:10.328: DCMResponseVariable [name={http://im.ca.com/normalizer}NormalizedPortInfo.Utilization, value=NULL ]
Jun 23 13:25:10.328: DCMResponseVariable [name={http://im.ca.com/normalizer}NormalizedPortInfo.FramesOut, value=77704 (Long) ]
Jun 23 13:25:10.328: DCMResponseVariable [name={http://im.ca.com/normalizer}NormalizedPortInfo.DiscardsIn, value=18649 (Long) ]
Jun 23 13:25:10.328: DCMResponseVariable [name={http://im.ca.com/normalizer}NormalizedPortInfo.UnicastIn, value=21337 (Long) ]
Jun 23 13:25:10.328: DCMResponseVariable [name={http://im.ca.com/normalizer}NormalizedPortInfo.BitsPerSecondIn, value=0.0 (Double) ]
Jun 23 13:25:10.328: DCMResponseVariable [name={http://im.ca.com/normalizer}NormalizedPortInfo.OperStatusPollable, value=1 (Long) ]
Jun 23 13:25:10.328: DCMResponseVariable [name={http://im.ca.com/normalizer}NormalizedPortInfo.PctDiscardsOut, value=26.51729640687738 (Double) ]
Jun 23 13:25:10.328: DCMResponseVariable [name={http://im.ca.com/normalizer}NormalizedPortInfo.FrameSizeOut, value=NULL ]
Jun 23 13:25:10.328: DCMResponseVariable [name={http://im.ca.com/normalizer}NormalizedPortInfo.UtilizationOut, value=NULL ]
Jun 23 13:25:10.328: DCMResponseVariable [name={http://im.ca.com/normalizer}NormalizedPortInfo.DiscardsOut, value=20605 (Long) ]
Jun 23 13:25:10.328: DCMResponseVariable [name={http://im.ca.com/normalizer}NormalizedPortInfo.PctErrorsIn, value=20.45250099370306 (Double) ]
Jun 23 13:25:10.328: DCMResponseVariable [name={http://im.ca.com/normalizer}NormalizedPortInfo.NonunicastOut, value=19101 (Long) ]
6. In this case, we are interested confirming our 20,605 value in step 4 with what Performance Management will import so we need to find the value listed for http://im.ca.com/normalizer}NormalizedPortInfo.DiscardsOut which in this case is:
Jun 23 13:25:10.328: DCMResponseVariable [name={http://im.ca.com/normalizer}NormalizedPortInfo.DiscardsOut, value=20605 (Long) ]
7. If this 20605 value is believed to be wrong, in this case, the incorrect data is being passed to Performance Management by the device itself. The recommendation moving forward would be to engage the Device Administrators and/or the Device Manufacturer for further assistance.