How to optimize distsrv probe performance when distributing superpackages
Article ID: 33940
Updated On:
Products
Issue/Introduction
Distribution of a superpackage containing 5 probes, to 50 robots seems too slow. The distribution took about hour.
There are two main options to improve distsrv performance. One option focuses on performance tuning for the CA UIM environment, especially the hub, and the other focuses on the distsrv probe configuration itself.
Environment
- Release: UIM 20.x or higher
- Component: UIMDSV
- distsrv
Resolution
Option 1:
Generally, prior to hub version 7.x, you can have the following queue bulk_size settings set anywhere from 100-999 with no issues. Below are some example settings used in MSP environments with a lot of message traffic. These settings can be modified using Raw Configure mode for the hub probe under the postroute section. Note that they should not be set blindly and it is highly recommended that any hub performance tuning should be done with assistance from CA UIM (NMS) Support personnel.
Sample bulk_size settings
hub bulk_size settings (local ATTACH queues)
- data_engine 1750
- alarm_enrichment 100
- nas 20
- audit 1 (is the default but it can be set from 100-999 if it's not draining well)
Sample bulk_size settings for GET queues configured on the primary hub/relay, configured for busy remote hubs:
- <remote_busy_hub1_qos> 1000
- <remote_busy_hub2_qos> 1000
- <remote_busy_hub1/relay1_alarms> 50
- <remote_busy_hub2/relay2_alarms> 50
Tip: As a rule of thumb and in practice for very busy remote hubs double the message receive rate and add 20% more to calculate the best bulk_size number.
Option 2:
To some degree, distsrv 'slowness' may be due to performance 'limitation' in the current design. File transfer of the distsrv works in a sequential synchronous transfer mode, where the files that need to be transmitted get divided up into chunks which are then sent and received/ack'd one by one, and chunk n+1 does not get put on the bus until chunk n has been ack'd.
To try and achieve higher throughput, the following distsrv configuration parameters should be considered.
<setup>
block_size = <number of bytes>
max_inst = <number of concurrent distributions>
forward_to_remote_hubs = <yes/no>
remote_max_inst = <number of distributions to forward during one session to the remote distribution server>
accept_remote_distributions = <yes/no>
use_local_archive = <yes/no>
</setup>
- block_size
Files are transferred in chunks of the configured size. The default setting for this one is quite low out of the box, so you can try increasing it, e.g., from 32k to 512k.
- max_inst
A thread is started for each concurrent installation. This setting determines how many concurrent threads will be started. The default setting is 10, but if the discovery server is running on a server with plenty of network bandwidth, you can try increasing it.
- forward_to_remote_hubs
This setting determines if distribution jobs are passed on to secondary distribution servers. Turn this on to offload the main distribution server and let the distribution servers on the robots' hub execute the task.
- remote_max_inst
Determines how many jobs to transfer to secondary distribution servers (on remote hubs) in one session. You should only need to increase this if the secondary distribution server finishes so quickly that their queues empty while there are still jobs on the primary distribution server waiting to be performed. Note also that this may in some situations need to be reduced if the primary distribution server is too congested.
- accept_remote_distributions
This option must be on, on the secondary distribution servers (it IS on by default).
- use_local_archive
This setting determines if the secondary hub distribution servers will request the packages from the Primary or use packages in their own local archive. If you want to turn this on, you should also forward packages and licenses from the Primary to the secondary distribution servers.
Note that all of the above settings can be modified via Raw Configure. To enter Raw Configure mode, select the distsrv probe and hold down the SHIFT key and then rt-click and Select Raw Configure.
Here are some of the settings that are exposed through the GUI:
Note that when you're editing the parameters using Raw Configure mode, you will find all of the configuration settings under the <setup> section which is displayed by default when you open the probe in that mode.
Additional Information
For more information please refer to the distsrv help documentation at:
Feedback
