Applicable versions: SaltStack Enterprise version 5.5 through 6.2
SaltStack Enterprise includes new background worker options to optimize performance for a range of deployment scenarios, since version 5.5. Background workers handle processes that run in the background, such as running jobs according to a given schedule, or syncing with an LDAP server.
New options for background workers are particularly useful for high-availability deployments. You can customize background worker settings to share background processes across multiple servers.
VMware Aria Automation Config 8.12.x
To define background worker options, modify the raas configuration file when you install or upgrade SaltStack Enterprise, or any time you want to improve the performance of your Enterprise deployment.
See the Enterprise download and installation page for full installation instructions.
See Commands for a list of commands that are useful for troubleshooting.
The following table describes available background worker options.
For recommended background worker settings, see Examples of background worker settings.
Option | Description |
combined_process |
Boolean to define whether both web server processes and background workers run on the server.
|
concurrency |
Number of worker processes. One unit represents one CPU core.
|
max_tasks |
Number of tasks a worker can complete until it restarts. |
max_memory |
Amount of memory in kB a worker can consume until it restarts.
|
Use the following commands to assist with troubleshooting. The -l
flag indicates debug mode.
Command | Description |
raas -l info |
Run both web server processes and background worker processes.
To run only web server processes, add the following to background_workers: combined_process: False |
raas -l info worker |
Run only worker processes. |
raas -l info beat |
Run only the beat process. The beat process initiates other processes at a set interval, distributing periodic tasks to workers. |
raas celery purge |
Purge the task queue. |
You can customize background worker settings to improve performance for a range of deployment scenarios. Multiple scenarios are described below, with recommended settings for each.
Note: In the following examples, num_processes
represents the number of web server processes.
The following example is a shared system where both web processes and background worker processes run concurrently on each server, supported by a load balancer. Each server includes 10 CPU cores. Processing power is distributed evenly across both types of processes.
/etc/raas/raas
On each server:
num_processes: 5 background_workers: combined_process: True concurrency: 5 max_tasks: 100000 max_memory: 1048576
The following example is a deployment that experiences a large amount of web server processes and relatively few background processes.
This scenario might be found, for example, in an environment that has many minions, and does not rely heavily on schedules.
/etc/raas/raas
On one server:
num_processes: 5 background_workers: combined_process: True concurrency: 5 max_tasks: 100000 max_memory: 1048576
On a second server:
num_processes: 10 background_workers: combined_process: False
The following example is a deployment that needs to support a large amount of background processes, and relatively few web server processes.
This scenario might be found, for example, in an environment that relies heavily on schedules. Each server includes 10 CPU cores.
/etc/raas/raas
On one server:
num_processes: 5 background_workers: combined_process: True concurrency: 5 max_tasks: 100000 max_memory: 1048576
On a second server:
background_workers: combined_process: False concurrency: 10
The following example is a multi-node system where web processes and background worker processes run on separate servers. Each server includes 10 CPU cores.
This configuration provides a simple framework for scaling SaltStack Enterprise without a load balancer, by moving your background work to a second server.
/etc/raas/raas
On one server:
num_processes: 10 background_workers: combined_process: False
On a second server:
background_workers: combined_process: False concurrency: 10