[EventQueueDispatcher] [EventQueueDispatcher] thread failed.

Products

• IT Management Suite

Issue/Introduction

You noticed this error in the NS logs multiple times during the day:

[EventQueueDispatcher] [EventQueueDispatcher] thread failed.

Value cannot be null.
Parameter name: path2
[System.ArgumentNullException @ mscorlib]
at System.IO.Path.Combine(String path1, String path2)
at Altiris.NS.ClientMessaging.EventQueueDispatcher.EventQueue.Assign(String deliverPath, ParameterizedThreadStart worker, DispatchSlot slot, EventRow row)
at Altiris.NS.ClientMessaging.EventQueueDispatcher.EventQueue.Pull(DispatchSlot slot, String deliverPath, ParameterizedThreadStart worker)
at Altiris.NS.ClientMessaging.EventQueueDispatcher.EventQueue.Pull(String deliverPath, ParameterizedThreadStart worker)
at Altiris.NS.ClientMessaging.EventQueueDispatcher.EventQueue.Push(String deliverPath, ParameterizedThreadStart worker, ICollection`1 rows)
at Altiris.NS.ClientMessaging.EventQueueDispatcher.DistributeForQueue(Int32 queueId, ICollection`1 rows)
at Altiris.NS.ClientMessaging.EventQueueDispatcher.LoadAndDistribute(EDistributeReason reason, Int32& completed)
at Altiris.NS.ClientMessaging.EventQueueDispatcher.DistributeQueues(EDistributeReason reason, Int32& completed)
at Altiris.NS.ClientMessaging.EventQueueDispatcher.DistributeQueues(EDistributeReason reason)
at Altiris.NS.ClientMessaging.EventQueueDispatcher.DistributeStep(EDistributeReason reason)
at Altiris.NS.ClientMessaging.EventQueueDispatcher.PerformWork()
at Altiris.Diagnostics.Threading.DiagnosticsThreadRunner.PerformWakeup()
at Altiris.Common.Threading.CommonThreadRunner.PerformWakeup()
at Altiris.NS.Threading.NsThreadRunner.PerformWakeup()
at Altiris.Diagnostics.Threading.DiagnosticsThreadRunner.PerformLoop(Thread me, Int32 version, WaitHandle[] waiters, Int32 terminatingEvents)

Exception logged from:
at Altiris.Diagnostics.Threading.DiagnosticsThreadRunner.Log(Int32, String, Exception)
at Altiris.Diagnostics.Threading.DiagnosticsThreadRunner.HandleException(Exception)
at Altiris.Diagnostics.Threading.DiagnosticsThreadRunner.PerformLoop(System.Threading.Thread, Int32, System.Threading.WaitHandle[], Int32)
at Altiris.Diagnostics.Threading.DiagnosticsThreadRunner.RunnerThreadProc()
at System.Threading.ThreadHelper.ThreadStart()

-----------------------------------------------------------------------------------------------------
Date: 6/24/2020 9:43:38 AM, Tick Count: 1923099515 (22.06:11:39.5150000), Size: 2.46 KB
Process: AeXSvc (33944), Thread ID: 231, Module: AeXSVC.exe
Priority: 1, Source: EventQueueDispatcher

Environment

ITMS 8.5

Cause

Stale NSEs entry records in the database.

The customer at one point moved and deleted the incoming NSEs from the EvtQueue folder into a temp folder. Then he tried to re-process them but the NSE entries in the database didn't match anymore.

The code portion where it fails:

slot.ProcessFile = slot.FileInLine ? row.File : Path.Combine(deliverPath, row.File);

actually should not fail, as the file "is not inline" (as the Path.Combine() executed) i.e. - the "File" property should contain the correct NSE file name.
But the Combine() fails exactly for the file name argument (path2) being null.  Something was wrong either with NSE registration (that saved NULL into DB) or with "pull" logic not being able to read file name from the tables.

Resolution

A simple test is to run the following query:

--For example, if I run this query about once every minute:

select min(id) as Oldest, max(id) as Newest
from EventQueueEntry

If you see that under the Oldest or Newest column the reference is not changing after refreshing the results a couple of times, it is most likely you have a NSE entry that is out of sync between what you have in the physical Queue folder and what the database is aware of.

In order to clear out those stale entries, please try the following:

1. Stop the Altiris services:
   • Altiris Support Service
   • Altiris Service 
   • Altiris Client Message Dispatcher
   • Altiris File Receiver

    

2. Stop the IIS service
   • World Wide Web Publishing Service (from the command line as administrator run IISRESET / STOP)

    

3. Clearing the event and merge tables with this SQL:
   
   

   --truncate the following tables

   truncate table eventqueue

   truncate table eventqueueentry

   truncate table [EventQueueEntryMetaData]

   truncate table eventqueueprocess --only if it has stale data

   truncate table eventqueuestatus --only if it has stale data

    

    

   --validate that they are empty

   select * from eventqueue

   select * from eventqueueentry

   select * from EventQueueEntryMetaData

   select * from EventQueueProcess

   select * from EventQueueStatus

   
   
   

4. Restart the SMP services:

   • Altiris Support Service
   • Altiris Service
   • Altiris Client Message Dispatcher
   • Altiris File Receiver
     
     
5. Start the IIS service

   World Wide Web Publishing Service (from the command line as administrator run IISRESET /START)

 

Additional Information

"Increasing Queue processing performance in SMP 8.5" (KB 172741)