A guide for benchmarking to help size your systems properly.
This guide was created to help customers of SaltStack Enterprise (SSE) benchmark their systems in a standard fashion that can be compared to a reference SSE system. This will help the SaltStack support staff recommend upgrades and troubleshoot performance bottlenecks.
For ease of use and setup, the Phoronix Test Suite is used to run selected benchmarks. You can download Phoronix from: https://www.phoronix-test-suite.com/?k=downloads
For network testing iperf3
will also need to be installed. This utility is usually available in your OS package manager.
When performing the benchmarks, you don’t necessarily have to run them on your production environment. You should run them on a machine that will emulate the same specs as the component you plan to run on it.
Note on large shared systems: If your system is large and doesn’t have dedicated RAM or CPUs, please consult your support contact. Some benchmarks may overuse the RAM or CPU, causing long benchmark times and conflicts with other devices sharing resources.
The reference system below is not necessarily recommended for every piece of SSE infrastructure but only used for a baseline comparison. For example, your Redis server can often have lower specs and a slower disk. Many times you’ll want a faster system for your Postgres DB if you are operating at a larger scale. SaltStack Customers have a variety of hardware and these benchmarks help determine the ability of your systems compared to a baseline. System recommendations will be made by support staff based on each client's results and based on the desired scale and use cases.
Reference System Specs:
Amazon AWS: m5d.2xlarge
Cores: 8
Memory: 32GB
Disk: 1 x 300 NVMe SSD
Bandwidth: Up to 10Gbps
Run the following benchmarks on a machine that is similar to the machine you will run the raas service on. The raas service runs a web server with background workers.
Benchmark | Test | Reference System |
nginx | --- | Avg: 27,224.89 Requests Per Second |
pybench | --- | Avg: 1370 Milliseconds |
sysbench | CPU | Avg: 7,307.63 Events Per Second |
sysbench | Memory | Avg: 8,908,310.27 Events Per Second |
To benchmark networking, install iperf3 on both the Postgres Server and RaaS Server.
iperf3 -s
iperf3 -c {pg-ip-address}
Reference System Results: 9.57 Gbits/sec
To benchmark networking, install iperf3
on both the Redis Server and RaaS Server.
iperf3 -s
iperf3 -c {redis-ip-address}
Reference System Results: 9.57 Gbits/sec
Run the following benchmarks on a machine that is similar to the machine on which you will run the Redis Service.
Benchmark | Test | Reference System |
redis | LPOP | Avg: 1,831,649.29 Requests Per Second |
SADD | Avg: 1,477,841.83 Requests Per Second | |
LPUSH | Avg: 1,208,712.50 Requests Per Second | |
GET | Avg: 1,763,672.13 Requests Per Second | |
SET | Avg: 1,342,488.21 Requests Per Second |
Run the following benchmarks on a machine that is similar to the machine on which you will run the Postgres Database.
Benchmark | Test | Reference System |
pgbench Note: this test takes a long time, usually at least 1hr+ | Scaling: On-Disk Test: Single Thread Mode: Read Only | Avg: 23379.73 TPS |
Scaling: On-Disk Test: Single Thread Mode: Read Write | Avg: 96.00 TPS | |
compress-pbzip2 | --- | Avg: 9.66 Seconds |
sysbench | CPU | Avg: 7,307.63 Events Per Second |
sysbench | Memory | Avg: 8,908,310.27 Events Per Second |
Run the following benchmarks on a machine that is similar to the machine on which you will run your Salt Master.
Benchmarks | Test | Reference System |
pybench | --- | Avg: 1370 Milliseconds |
iozone | Record Size: 64Kb File Size: 512MB Disk Test: Read Performance | Avg: 6155.72 MB/s |
iozon | Record Size: 64Kb File Size: 512MB Disk Test: Write Performance | Avg: 493.56MB/s |
compress-pbzip2 | --- | Avg: 9.66 Seconds |
sysbench | CPU | Avg: 7,307.63 Events Per Second |
sysbench | Memory | Avg: 8,908,310.27 Events Per Second |
To benchmark networking, install iperf3
on both the Salt Master and RaaS Server.
iperf3 -s
iperf3 -c {raas-ip-address}
Reference System Results: 9.57 Gbits/sec