Benchmarking Guide For SaltStack Enterprise
search cancel

Benchmarking Guide For SaltStack Enterprise

book

Article ID: 312992

calendar_today

Updated On:

Products

VMware Aria Suite

Issue/Introduction

A guide for benchmarking to help size your systems properly.

Introduction 

This guide was created to help customers of SaltStack Enterprise (SSE) benchmark their systems in a standard fashion that can be compared to a reference SSE system. This will help the SaltStack support staff recommend upgrades and troubleshoot performance bottlenecks.

For ease of use and setup, the Phoronix Test Suite is used to run selected benchmarks. You can download Phoronix from: https://www.phoronix-test-suite.com/?k=downloads

For network testing iperf3 will also need to be installed. This utility is usually available in your OS package manager.

When performing the benchmarks, you don’t necessarily have to run them on your production environment. You should run them on a machine that will emulate the same specs as the component you plan to run on it.

Note on large shared systems: If your system is large and doesn’t have dedicated RAM or CPUs, please consult your support contact. Some benchmarks may overuse the RAM or CPU, causing long benchmark times and conflicts with other devices sharing resources.

Reference system 

The reference system below is not necessarily recommended for every piece of SSE infrastructure but only used for a baseline comparison. For example, your Redis server can often have lower specs and a slower disk. Many times you’ll want a faster system for your Postgres DB if you are operating at a larger scale. SaltStack Customers have a variety of hardware and these benchmarks help determine the ability of your systems compared to a baseline. System recommendations will be made by support staff based on each client's results and based on the desired scale and use cases.

Reference System Specs:
Amazon AWS: m5d.2xlarge
Cores: 8
Memory: 32GB
Disk: 1 x 300 NVMe SSD
Bandwidth: Up to 10Gbps

SSE server benchmarks 

Run the following benchmarks on a machine that is similar to the machine you will run the raas service on. The raas service runs a web server with background workers.
 

BenchmarkTestReference System
nginx---Avg: 27,224.89 Requests Per Second
pybench---Avg: 1370 Milliseconds
sysbenchCPUAvg: 7,307.63
Events Per Second
sysbenchMemoryAvg: 8,908,310.27
Events Per Second

RaaS server to Postgres network benchmark 

To benchmark networking, install iperf3 on both the Postgres Server and RaaS Server. 

  1. On the Postgres Server run: iperf3 -s 
  2. On the RaaS Server run: iperf3 -c {pg-ip-address}

Reference System Results: 9.57 Gbits/sec
 

RaaS server to Redis network benchmark 

To benchmark networking, install iperf3 on both the Redis Server and RaaS Server.

  1. On the Redis Server run: iperf3 -s 
  2. On the RaaS Server run: iperf3 -c {redis-ip-address}

Reference System Results: 9.57 Gbits/sec

 

Redis server benchmarks 

Run the following benchmarks on a machine that is similar to the machine on which you will run the Redis Service.

BenchmarkTestReference System
redisLPOPAvg: 1,831,649.29 Requests Per Second
 SADDAvg: 1,477,841.83 Requests Per Second
 LPUSHAvg: 1,208,712.50 Requests Per Second
 GETAvg: 1,763,672.13 Requests Per Second
 SETAvg: 1,342,488.21 Requests Per Second

Postgres server benchmarks 

Run the following benchmarks on a machine that is similar to the machine on which you will run the Postgres Database.

BenchmarkTestReference System
pgbench
Note: this test takes a long time, usually at least 1hr+
Scaling: On-Disk
Test: Single Thread
Mode: Read Only
Avg: 23379.73 TPS
 Scaling: On-Disk
Test: Single Thread
Mode: Read Write
Avg: 96.00 TPS
compress-pbzip2---Avg: 9.66 Seconds
sysbenchCPUAvg: 7,307.63
Events Per Second
sysbenchMemoryAvg: 8,908,310.27
Events Per Second

Salt Master benchmarks 

Run the following benchmarks on a machine that is similar to the machine on which you will run your Salt Master.

BenchmarksTestReference System
pybench---Avg: 1370 Milliseconds
iozoneRecord Size: 64Kb
File Size: 512MB
Disk Test: Read Performance
Avg: 6155.72 MB/s
iozonRecord Size: 64Kb
File Size: 512MB
Disk Test: Write Performance
Avg: 493.56MB/s
compress-pbzip2---Avg: 9.66 Seconds
sysbenchCPUAvg: 7,307.63
Events Per Second
sysbenchMemoryAvg: 8,908,310.27
Events Per Second

Salt Master to RaaS Server network benchmark

To benchmark networking, install iperf3 on both the Salt Master and RaaS Server.

  1. On the RaaS Server run: iperf3 -s 
  2. On the Salt Master run: iperf3 -c {raas-ip-address}

Reference System Results: 9.57 Gbits/sec



Environment

VMware Aria Automation Config 8.12.x