SDC TrapX Performance and sizing in large environments

Products

Network Observability Spectrum

Issue/Introduction

What are the TrapExploder processing limitations?

Need to determine TrapX (TrapExploder) role in a new environment.

Has there been performance testing against TrapX? Is there a sizing tool for it?

If it was tested what threshold has been performed? Can it handle a large # of traps?

Environment

All supported Network Observability DX NetOps Spectrum SDC TrapX (TrapExploder) environments

Resolution

There is no formal sizing tool available for TrapX at this time. It's processing rate depends on the machine specifications like CPU, RAM, Processor speed and the type of filtering implemented. Additionally the trap rate for SNMPv3 traps can play a major role in performance. Higher rates of SNMPv3 traps vs SNMPv1/v2c traps will require higher processing requirements.

The number of traps per second TrapX (also known as TrapExploder) can handle and process is beyond what a single Spectrum Landscape server can process. This is with a single TrapX instance running on one small system in a large environment.

The amount of hardware required to process traps is significantly less than what a SpectroSERVER requires for trap processing.

It is worth noting TrapX is used for processing traps by filtering them out and dropping them, among other work. This is very different from the work a SpectroSERVER must perform which receives, expands, interprets and ties a trap to a discovered and managed Spectrum model for alerting. A much more resource intensive process than the work TrapX is performing.

For example if we take a TrapX system that has 2 CPU cores and 2GB of RAM, it can handle 1000's of traps per second. However a Spectrum SpectroSERVER itself can only handle about 600 traps per second at a maximum rate. It would also require 4 CPU cores and 16GB RAM, at a minimum, to accomplish that task.

Field experience from users with many years of TrapX use recommend deployment on a single system with 2 cores and 4GB RAM minimum. This should be more than adequate to handle any incoming trap loads on the TrapX side. It would also offset possible impact from large trap loads hitting the Spectrum server through, for example, removing lots of unauthenticated traps and others that aren't wanted. The fewer unwanted traps passed to Spectrum for processing, the better the SpectroSERVER will perform.

Concerning SNMP type levels, there is a performance hit when dealing with SNMP v3 traps as opposed to v2c or v1. The reason for this is to do with the handling of encryption. However, with current modern CPUs (that have crypto accelerators built into them), the impact is not really noticeable. It is useful to know the trap load expected between SNMPv3 and SNMPv1/v2c.

Additional Information

For performance measurements review the Grafana based Spectrum Performance reports and dashboards.
- They have the ability to provide traps per second rates from the Spectrum SpectroSERVER perspective.
- This does not display traps per second from TrapX, only traps Spectrum is processing.
In addition to the Grafana solution performance events are available on the SpectroSERVER to check traps per second.
- This can be accessed via the perf collector tool, or through the SSperformance application model on each landscape.
- There would be additional data available there with an SNMP agent on the SpectroSERVER.
Spectrum TrapInsight Dashboard View documentation topic
SDC TrapX Support documentation topic covering install, configuration and usage.
There are no REST interface or APIs available for TrapX.
TrapX is not able to be configured through Spectrum REST APIs.
TrapX can integrate with most SNMP trap receivers.
Want to track processed trap counts? Maybe gathering them on a regular basis, every 12-24 hours for example, to track the rates in a spreadsheet or something similar for reporting?
- Monitor the Traps_Processed attribute on the VNM model representing the SS processing the traps.
- The attribute value represents traps processed since the SS was last restarted.
- SS restarts reset the attribute value.
- To query the values via CLI for scripting use:
  - Find the VNM model in the OC UI. Note it's Model_Handle value from the Attributes tab.
    - It's normally the same as the landscape handle. For example it's 0x1000000 in most single SS systems.
  - Go to the $SPECROOT/vnmsh directory as the Spectrum install owner.
  - Run "./connect" to connect to the Spectrum CLI.
  - Run this for the VNM model using your landscapes VNM model Model_Handle attribute value for the mh value.
    - ./show attributes mh=0x1000000 | grep Traps_Processed