Customers deploying Security Intelligence (previously known as NSX Intelligence) have difficulty evaluating the number of worker nodes required for their environment. Incorrect sizing of worker nodes can result in instability - resulting in storage, memory, and CPU alarms, and improper functionality of enabled features.
This tool is used to estimate the number of worker nodes required to operate Security Intelligence smoothly.
This tool requires NSX Application Platform version 4.2.0 or above and NSX version 4.2.0 or above. This tool will not work or provide any useful output if either of the versions are below 4.2.0.
Security Intelligence deployment with insufficient resources and can lead to multiple issues as mentioned below.
Customers are advised to use the sizing tool referenced in this KB to estimate the number of worker nodes required for their environment. Here is the recommended workflow to install and use this tool.
1. Install the NSX Application Platform and allow the system to run and collect flow stats for at least seven days.
2. Download and install the sizing tool referenced in this KB.
a. Go to Broadcom NSX 4.2 download page and maneuver to Drivers and Tool tab.
b. Search for Security Intelligence Sizing Tool and expand to get the download icon. Download the package to your local storage (screenshot below).
c. Copy the security_intelligence_sizing package to any of the NSX Manager nodes in the cluster using the command below.
# scp security_intelligence_sizing root@<nsx_manager_IP>:/opt/vmware/bin
b. Login to NSX Manager as root and change the permission of the file using the command line below.
# chmod +x /opt/vmware/bin/security_intelligence_sizing
3. Run the sizing tool
a. To obtain a list of options, use the following command.
# security_intelligence_sizing –-help
b. Typical usage of the tool involves invoking it with the following command; the --verbose is optional.
# security_intelligence_sizing --manager localhost --username admin –-verbose
c. The tool will then prompt for the password for username "admin"; provide the admin password of the NSX Manager.
d. Here is one sample output where the number of worker nodes recommended is within the supported Config Max limit.
root@nsx-mgr-0:~# security_intelligence_sizing --manager localhost --username admin --verbose
2024-07-26 22:34:27 - DEBUG - NSX username: admin
2024-07-26 22:34:27 - DEBUG - NSX Manager: localhost
2024-07-26 22:34:27 - DEBUG - Using percentage for internal flows --internal_flows: 70
2024-07-26 22:34:27 - DEBUG - Using percentage for unique flows per hour --unique_flows: 15
2024-07-26 22:34:27 - DEBUG - Using NAPP raw flow capacity per compute instance per second --rawflow_capacity_per_instance_per_sec: 1,000
2024-07-26 22:34:27 - DEBUG - Using NAPP over flow capacity per compute instance per second --overflow_capacity_per_instance_per_sec: 800
2024-07-26 22:34:27 - DEBUG - Using NAPP disk size in GB per storage instance --historical_disk_size: 128
2024-07-26 22:34:27 - DEBUG - Using NAPP flow size in bytes --flow_size_in_bytes: 200
2024-07-26 22:34:27 - INFO - 24 Transport Nodes detected among 7 cluster(s) and standalone host(s).
This could take a while...
2024-07-26 22:34:27 - DEBUG - Extracting total flow metrics...
2024-07-26 22:34:39 - DEBUG - Extracting 5-minute interval flow metrics…
2024-07-26 22:34:50 - security_intelligence_sizing - Average 5-minute interval burst of correlated flows: 520,000
2024-07-26 22:34:50 - security_intelligence_sizing - Flow compute instance(s) required: 2
2024-07-26 22:34:50 - security_intelligence_sizing - Estimated number of correlated flows aggregated over 30 days: 833,040,000
2024-07-26 22:34:50 - security_intelligence_sizing - Flow storage instance(s) required: 1
2024-07-26 22:34:50 - security_intelligence_sizing - Minimum number of worker nodes required for this version of Security Intelligence: 4
2024-07-26 22:34:50 - security_intelligence_sizing - Maximum number of worker nodes supported in this version of Security Intelligence: 10
2024-07-26 22:34:50 - security_intelligence_sizing - Worker nodes recommended for this environment: 6
e. Here is another sample output where the recommended number of worker nodes exceeds the supported Config Max limit. In this case, the tool will report the flow statistics for each cluster and individual host.
root@nsx-mgr-0:~# security_intelligence_sizing --manager localhost --username admin --verbose
2024-07-26 22:34:27 - DEBUG - NSX username: admin
2024-07-26 22:34:27 - DEBUG - NSX Manager: localhost
2024-07-26 22:34:27 - DEBUG - Using percentage for internal flows --internal_flows: 70
2024-07-26 22:34:27 - DEBUG - Using percentage for unique flows per hour --unique_flows: 15
2024-07-26 22:34:27 - DEBUG - Using NAPP raw flow capacity per compute instance per second --rawflow_capacity_per_instance_per_sec: 1,000
2024-07-26 22:34:27 - DEBUG - Using NAPP over flow capacity per compute instance per second --overflow_capacity_per_instance_per_sec: 800
2024-07-26 22:34:27 - DEBUG - Using NAPP disk size in GB per storage instance --historical_disk_size: 128
2024-07-26 22:34:27 - DEBUG - Using NAPP flow size in bytes --flow_size_in_bytes: 200
2024-07-26 22:34:27 - INFO - 324 Transport Nodes detected among 7 cluster(s) and standalone host(s). This could take a while...
2024-07-26 22:34:27 - DEBUG - Extracting total flow metrics...
2024-07-26 22:34:39 - DEBUG - Extracting 5-minute interval flow metrics...
2024-07-26 22:34:50 - DEBUG - Average raw flows per hour over last 7 days: 19,663,105
2024-07-26 22:34:50 - DEBUG - Average 5-minute interval burst of raw flows over last 7 days: 2,191,957
2024-07-26 22:34:50 - DEBUG - Average 5-minute interval burst of correlated flows: 1,424,772
2024-07-26 22:34:50 - DEBUG - Flow compute instance(s) required: 8
2024-07-26 22:34:50 - DEBUG - Estimated number of correlated flows aggregated over 30 days: 1,706,265,870
2024-07-26 22:34:50 - DEBUG - Flow storage instance(s) required: 4
2024-07-26 22:34:50 - DEBUG - Minimum number of worker nodes required for this version of Security Intelligence: 4
2024-07-26 22:34:50 - DEBUG - Maximum number of worker nodes supported in this version of Security Intelligence: 10
2024-07-26 22:34:50 - INFO - The volume of flows in this environment requires 2 more worker nodes than the supported limit. Below is a csv output that should help you identify a subset of clusters or standalone hosts to include in your Security Intelligence deployment
8 <------------------------
Cluster Name,5-minute Interval Flows,Average Flows Per Hour
Tenant-Cluster-0,9796,109458
Tenant-Cluster-1,9722,111858
SIM_Cluster,6872,56241
# security_intelligence_sizing --manager localhost --username admin –-verbose –-activated_only
4. If the number of worker nodes currently existing in the environment is less than what the sizing tool recommends, then deploy additional worker nodes as recommended by the tool.
STEP1: Do one of the following depending on how NAPP was deployed.
STEP2: At the System tab 1 in NSX Application Platform UI 2 under the Actions 3 choose the scale-out 4 as below.
STEP3: Select Analytics, Messaging, and Data Storage (if applicable) services and click the SCALE OUT radio button
Advanced Options
The tool implements a default value of traffic that is considered internal (within your NSX-managed workloads) and percentage of traffic that is unique within the last hour (non-repeated).
If you suspect your data center to have less internal traffic, you can adjust the internal percentage by using option --internal_flow_percent.
Example:
# security_intelligence_sizing --manager localhost --username admin –-verbose --internal_flow_percent 50
If your deployment includes a VDI environment, a higher percentage of unique traffic might be expected; please adjust the unique percentage accordingly using option --unique_flow_percent.
Example:
# security_intelligence_sizing --manager localhost --username admin –-verbose --unique_flow_percent 25