Security Intelligence Sizing Tool
search cancel

Security Intelligence Sizing Tool

book

Article ID: 373793

calendar_today

Updated On:

Products

VMware vDefend Firewall VMware vDefend Firewall with Advanced Threat Prevention

Issue/Introduction

Customers deploying Security Intelligence (previously known as NSX Intelligence) have difficulty evaluating the number of worker nodes required for their environment. Incorrect sizing of worker nodes can result in instability - resulting in storage, memory, and CPU alarms, and improper functionality of enabled features.

This tool is used to estimate the number of worker nodes required to operate Security Intelligence smoothly.

Environment

This tool requires NSX Application Platform version 4.2.0 or above and NSX version 4.2.0 or above. This tool will not work or provide any useful output if either of the versions are below 4.2.0.

Cause

Security Intelligence deployment with insufficient resources and can lead to multiple issues as mentioned below.

  • Alerts and Alarms indicating failures
  • Inability to ingest incoming flows
  • Inability to process and store incoming flow metrics data, events and configuration updates
  • Failure to provide accurate recommendations

Resolution

Customers are advised to use the sizing tool referenced in this KB to estimate the number of worker nodes required for their environment. Here is the recommended workflow to install and use this tool.

1. Install the NSX Application Platform and allow the system to run and collect flow stats for at least seven days.
2. Intelligence Sizing Tool depends on Metrics vertical and not  NSX Intelligence vertical. Activating Intelligence to run Sizing toll not mandatory.

Note:  With Intelligence not activated/deployed, it would assume all clusters and hosts will be enabled for data collection. Incase, Intelligence already activated/deployed, then customers may choose clusters by activate/deactivate under System tab->NSX Intelligence settings, and select as per choice and run the sizing tool.

2. Download and install the sizing tool referenced in this KB.

a. Go to Broadcom NSX 4.2 download page and maneuver to Drivers and Tool tab.

b. Search for Security Intelligence Sizing Tool  and expand to get the download icon. Download the package to your local storage (screenshot below). 

c. Copy the security_intelligence_sizing package to any of the NSX Manager nodes in the cluster using the command below. 

# scp security_intelligence_sizing root@<nsx_manager_IP>:/opt/vmware/bin

b. Login to NSX Manager as root and change the permission of the file using the command line below.

# chmod +x /opt/vmware/bin/security_intelligence_sizing

3. Run the sizing tool

a. To obtain a list of options, use the following command.

# security_intelligence_sizing –-help

b. Typical usage of the tool involves invoking it with the following command; the --verbose is optional.

# security_intelligence_sizing --manager localhost --username admin –-verbose

c. The tool will then prompt for the password for username "admin"; provide the admin password of the NSX Manager. 

d. Here is one sample output where the number of worker nodes recommended is within the supported Config Max limit.

root@nsx-mgr-0:~# security_intelligence_sizing --manager localhost --username admin --verbose
2024-07-26 22:34:27 - DEBUG - NSX username: admin
2024-07-26 22:34:27 - DEBUG - NSX Manager: localhost
2024-07-26 22:34:27 - DEBUG - Using percentage for internal flows --internal_flows: 70
2024-07-26 22:34:27 - DEBUG - Using percentage for unique flows per hour --unique_flows: 15
2024-07-26 22:34:27 - DEBUG - Using NAPP raw flow capacity per compute instance per second --rawflow_capacity_per_instance_per_sec: 1,000
2024-07-26 22:34:27 - DEBUG - Using NAPP over flow capacity per compute instance per second --overflow_capacity_per_instance_per_sec: 800
2024-07-26 22:34:27 - DEBUG - Using NAPP disk size in GB per storage instance --historical_disk_size: 128
2024-07-26 22:34:27 - DEBUG - Using NAPP flow size in bytes --flow_size_in_bytes: 200
2024-07-26 22:34:27 - INFO - 24 Transport Nodes detected among 7 cluster(s) and standalone host(s).

This could take a while...

2024-07-26 22:34:27 - DEBUG - Extracting total flow metrics...
2024-07-26 22:34:39 - DEBUG - Extracting 5-minute interval flow metrics…

2024-07-26 22:34:50 - security_intelligence_sizing - Average 5-minute interval burst of correlated flows: 520,000

2024-07-26 22:34:50 - security_intelligence_sizing - Flow compute instance(s) required: 2

2024-07-26 22:34:50 - security_intelligence_sizing - Estimated number of correlated flows aggregated over 30 days: 833,040,000

2024-07-26 22:34:50 - security_intelligence_sizing - Flow storage instance(s) required: 1

2024-07-26 22:34:50 - security_intelligence_sizing - Minimum number of worker nodes required for this version of Security Intelligence: 4

2024-07-26 22:34:50 - security_intelligence_sizing - Maximum number of worker nodes supported in this version of Security Intelligence: 10

2024-07-26 22:34:50 - security_intelligence_sizing - Worker nodes recommended for this environment: 6

e. Here is another sample output where the recommended number of worker nodes exceeds the supported Config Max limit. In this case, the tool will report the flow statistics for each cluster and individual host.

root@nsx-mgr-0:~# security_intelligence_sizing --manager localhost --username admin --verbose

2024-07-26 22:34:27 - DEBUG - NSX username: admin

2024-07-26 22:34:27 - DEBUG - NSX Manager: localhost

2024-07-26 22:34:27 - DEBUG - Using percentage for internal flows --internal_flows: 70

2024-07-26 22:34:27 - DEBUG - Using percentage for unique flows per hour --unique_flows: 15

2024-07-26 22:34:27 - DEBUG - Using NAPP raw flow capacity per compute instance per second --rawflow_capacity_per_instance_per_sec: 1,000

2024-07-26 22:34:27 - DEBUG - Using NAPP over flow capacity per compute instance per second --overflow_capacity_per_instance_per_sec: 800

2024-07-26 22:34:27 - DEBUG - Using NAPP disk size in GB per storage instance --historical_disk_size: 128

2024-07-26 22:34:27 - DEBUG - Using NAPP flow size in bytes --flow_size_in_bytes: 200

2024-07-26 22:34:27 - INFO - 324 Transport Nodes detected among 7 cluster(s) and standalone host(s). This could take a while...

2024-07-26 22:34:27 - DEBUG - Extracting total flow metrics...

2024-07-26 22:34:39 - DEBUG - Extracting 5-minute interval flow metrics...

2024-07-26 22:34:50 - DEBUG - Average raw flows per hour over last 7 days: 19,663,105

2024-07-26 22:34:50 - DEBUG - Average 5-minute interval burst of raw flows over last 7 days: 2,191,957

2024-07-26 22:34:50 - DEBUG - Average 5-minute interval burst of correlated flows: 1,424,772

2024-07-26 22:34:50 - DEBUG - Flow compute instance(s) required: 8

2024-07-26 22:34:50 - DEBUG - Estimated number of correlated flows aggregated over 30 days: 1,706,265,870

2024-07-26 22:34:50 - DEBUG - Flow storage instance(s) required: 4

2024-07-26 22:34:50 - DEBUG - Minimum number of worker nodes required for this version of Security Intelligence: 4

2024-07-26 22:34:50 - DEBUG - Maximum number of worker nodes supported in this version of Security Intelligence: 10

2024-07-26 22:34:50 - INFO - The volume of flows in this environment requires 2 more worker nodes than the supported limit. Below is a csv output that should help you identify a subset of clusters or standalone hosts to include in your Security Intelligence deployment

8 <------------------------

Cluster Name,5-minute Interval Flows,Average Flows Per Hour

Tenant-Cluster-0,9796,109458

Tenant-Cluster-1,9722,111858

SIM_Cluster,6872,56241

      • In this scenario, use the CSV snippet under 8 <------------------------, in the output above, to determine which clusters to select.
      • Once the clusters and stand-alone hosts are determined, use the NSX Intelligence UI to enable only select clusters. At the System tab 1 in NSX Intelligence 2 settings, Select 3 some clusters or stand-alone hosts and hit the Deactive 4 button as below. Please note that NSX Intelligence needs to be activated to see this screen.

      • Next, run the tool with the --activated_only option to compute the sizing only for the enabled clusters.

# security_intelligence_sizing --manager localhost --username admin –-verbose –-activated_only

4. If the number of worker nodes currently existing in the environment is less than what the sizing tool recommends, then deploy additional worker nodes as recommended by the tool. 

 STEP1: Do one of the following depending on how NAPP was deployed.

      • If Automation Appliance was used, then increase the number of worker nodes in the NAPP Automation Appliance UI.
      • If Automation Appliance was not used during initial deployment, work with the cluster administrator to add more nodes.

STEP2: At the System tab 1 in NSX Application Platform UI 2 under the Actions 3 choose the scale-out 4 as below.

STEP3: Select Analytics, Messaging, and Data Storage (if applicable) services and click the SCALE OUT radio button

 

 

 



Additional Information

Advanced Options

The tool implements a default value of traffic that is considered internal (within your NSX-managed workloads) and percentage of traffic that is unique within the last hour (non-repeated).

  • Default percentage of internal traffic is 70%
  • Default percentage of unique traffic is 15%


If you suspect your data center to have less internal traffic, you can adjust the internal percentage by using option --internal_flow_percent.

Example:

# security_intelligence_sizing --manager localhost --username admin –-verbose --internal_flow_percent 50

If your deployment includes a VDI environment, a higher percentage of unique traffic might be expected; please adjust the unique percentage accordingly using option --unique_flow_percent.

Example:

# security_intelligence_sizing --manager localhost --username admin –-verbose --unique_flow_percent 25

Powered by Wolken Software