Smarts IP Manager (Smarts IP) :How to verify the cause of high IfOutDiscardsRate alerts?; Why are we getting high IfOutDiscardsRate alerts?

Products

• VMware Smart Assurance

Issue/Introduction

Symptoms:




Why are we getting alerts for high IfOutDiscardsRate?  How is this rate calculated?

Environment

VMware Smart Assurance - SMARTS

Resolution

IfOutDiscardsRate is the rate of outbound packets, expressed in packets per second, chosen to be discarded even though no errors had been detected.

In order to investigate why high IfOutDiscardsRate is generated, we must gather the below data :

1. roll the logs : (./dmctl -s <server_name> exec roll_log)

2. Properly subscribe to the ifOutDiscardsRate and ifOutDiscards simultaneously.  Execute the below command:

./sm_adapter -s <server_name> --subscribeProp="<Subscription_details>::<Port_details>" | tee

For example, please refer to the below command ( ifMIB-PORT-TYSAGR01/1.29 will be used in upcoming computation ) :

./sm_adapter -s <server_name> --subscribeProp="NetworkAdapter_PerformanceData_MIB2_ifMIB::I-NetworkAdapter_PerformanceData_ifMIB-PORT-TYSAGR01/1.29::ifOutDiscards/v" --subscribeProp="NetworkAdapter_PerformanceData_MIB2_ifMIB::I-NetworkAdapter_PerformanceData_ifMIB-PORT-TYSAGR01/1.29::ifOutDiscards/v" |tee

With this data, we should be able to determine the root cause.
--------------------------------------------------------------------------------------------------------------------------------------------------------------

We compute the ifOutDiscardsRate as follows:

refine computed ifOutDiscardsRate = rate(ifOutDiscards, PollingInterval) else 0;

The rate definition consists of A & T, where :

A equals attribute 
T equals time interval 

The term rate (A,T), returns the rate at which A changed during the last T seconds. The time interval may be an expression that evaluates to a numeric value. 


You can begin evaluating  the logs based on the polling data extracted, for example 

1. 1st polling cycle value:
1434532030|ATTRIBUTE_CHANGE|NetworkAdapter_PerformanceData_ifMIB|I-NetworkAdapter_PerformanceData_ifMIB-PORT-TYSAGR01/1.29|ifOutDiscards|10490|

2. 2nd polling cycle value:
1434532270|ATTRIBUTE_CHANGE|NetworkAdapter_PerformanceData_ifMIB|I-NetworkAdapter_PerformanceData_ifMIB-PORT-TYSAGR01/1.29|ifOutDiscards|20980|

3. Rate computation as follows:
CurrentPollingCycleValue - PreviousPollingCycleValue / polling interval

Where are these values?
-CurrentPollingCycleValue= 1st polling cycle ifOutDiscards 10490
-PreviousPollingCycleValue= 2nd polling cycle ifOutDiscards 20980
-Polling interval = the difference between 1434532030 and 1434532270 = 240

i.e (20980 - 10490) / 240 = 43.70833333333333 // which is the ifOutDiscardsRate which is fine.

4. Now the next successive polling cycle, this is the value observed.
1434532510|ATTRIBUTE_CHANGE|NetworkAdapter_PerformanceData_ifMIB|I-NetworkAdapter_PerformanceData_ifMIB-PORT-TYSAGR01/1.29|ifOutDiscards|10490|

5. Now substituting, the value to the rate function, we have:
(10490 - 20980) / 240 = 1.78957E+07 // negative ifOutDiscardsRate value.

From the above computation we see that previous value is 10490 and the successive value is 20980. Hence ifoutdiscards is a counter attribute which means that it each successive value should be incremental and cannot be decreasing.

Kindly note if there are abrupt change in values, you will need to consult the device vendor to get clarification on the MIB definition.