Performance issues with queue processing
search cancel

Performance issues with queue processing

book

Article ID: 151968

calendar_today

Updated On:

Products

IT Management Suite

Issue/Introduction

The following items are problems associated with the aggressive number of threads processing queued event (NSE) on a default installation of the Notification Server:

     1. SQL Deadlocks
     2. Slow processing of NSEs in the file queues (Evtqueue, Evtqfast)
     3. CPU utilization spiking for the AeXSvc.exe (Altiris Service)

Cause

Aggressive multi-threading configuration can actually cause performance issues on the Notification Server.

  • The Evtqueue, which handles most inventory is set by default to use 3 concurrent threads.
  • The Evtqfast, which handles all small NSE's is set by default to use 10 concurrent threads.

Resolution

Before modifying any settings, please ensure that the total number of files under all the bad NSE folders is less than 10,000.   File system operations will have a direct negative impact on the speed of NSE processing if a large number of files are present in these folders.  This is due to the need to ensure that the rejected nse file has a unique name for the target bad folder.  The NS can't start processing the next nse until the previous file has been moved or deleted.

Note: Dispatcher thread tuning is entirely dependent upon the capacity of the SQL server to process incoming data.  In most environments, decreasing the number of concurrent threads actually yields better results, due to the reduced occurrences of SQL blocking and or deadlocking.

Whenever possible, please use the NS Configurator tool to make these type of changes (to avoid mistakes, which leads to data corruption).  The installation msi package is found under the Altiris\Diagnostics directory.  After installation open the Start Menu > Altiris > Tools > NS Configurator > Search for "MaxConcurrent"


In the Coresettings.config file (located under installpath\Program Files\Altiris\Notification Server\Config\), there are two settings (case sensitive) that control how many threads the dispatcher service will use on the specified queue:

     1. <customSetting key="MaxConcurrentFastMsgs" type="local" value="10" />
     2. <customSetting key="MaxConcurrentSlowMsgs" type="local" value="3" />
     3. The Evtqslow cannot be changed, and has a default of 1 thread

The following items will assist in determining what these settings should be set to:

     •  For the EvtQueue (MaxConcurrentSlowMsgs).  The value of this setting should be left at the default for most environments.  A direct cause of SQL deadlocks during dataloader operations is a result of increasing the number of these threads.  For under-performing SQL servers, this value should be lowered to a minimum of 1.  The most popular value for resolving deadlocks is 2.

     •  For the Evtqfast.  The value of this setting should be left at the default for most environments.  A direct cause of SQL deadlocks during dataloader operations is a result of increasing the number of these threads.  For under-performing SQL servers, this value should be lowered to a minimum of 4.  The most popular value for resolving deadlocks is 5.


Applies To
Notification Server Service Pack 3 with default settings.