SiteTransfer register operations do not commit transactions
search cancel

SiteTransfer register operations do not commit transactions

book

Article ID: 264600

calendar_today

Updated On:

Products

IT Management Suite

Issue/Introduction

The customer reported the following symptoms:

  • Slow event processing
  • High SQL CPU usage

It was discovered that "SiteTransfer" requests write into the database as a transaction which never committed.

Eventually, such transactions fail because of remote query timeouts.

Periodically, transaction logs become full causing all other transactions to fail.

 

From the NS logs, this seems to be the sequence of messages:

Entry 1:

DatabaseContext validation failed.
 Context stack: [#0] 257:2370b850-5e2a-4e0c-abb3-ac307009f7b6
 Current: [2/257/0] {TranOwner,ReadCommitted}, t='Rolledback' ([05fdc407ae8e44649367b0ea91f4dbfa]), s=[#0] 257:2370b850-5e2a-4e0c-abb3-ac307009f7b6, Altiris.NS.ContextManagement.AdminDatabaseContext

Incorrect database context object.
   [InvalidDatabaseContextException]

Exception logged from:
   at Altiris.Diagnostics.Logging.EventLog.ReportException(int, string, string, Exception, string)
   at Altiris.Database.DatabaseContext<>.LogError(string, Exception)
   at Altiris.Database.DatabaseContext<>.ValidateDisposingContext(bool)
   at Altiris.Database.DatabaseContext<>.Dispose()
   at Altiris.NS.DataAccessLayer.Implementation.Altiris_NS_SiteWideSiteTransferDal.Delete(Guid, Guid, Guid)
   at Altiris.NS.SiteWide.SiteTransferRequestHelper+<>c__DisplayClass15_0.<Serve>b__1(IDatabaseContext)
   at Altiris.NS.ContextManagement.Altiris.NS.ContextManagement.AdminDatabaseContext+<>c__DisplayClass26_0<>.<PerformTransactedDeadlockRetry>b__0(IDatabaseContext)
   at Altiris.Database.RetryActionRequest<>.Perform(bool)
   at Altiris.Database.DatabaseContext<>.PerformWithDeadlockRetryHelper(int, int, bool, Getter<IDatabaseContext>, Action<IDatabaseContext>, Action<DeadlockRetryArgs>, bool, string, string)
   at Altiris.NS.SiteWide.SiteTransferRequestHelper.Serve(SiteTransferRequest, out string)
   at Altiris.NS.StandardItems.AgentManagement.Communication.Handlers.AgentActionSiteTransferHandler.Handle(INSAgentConnection, IAgentAction<string>)
   at System.Threading.Tasks.ContinuationResultTaskFromTask<>.InnerInvoke()
   at System.Threading.Tasks.Task.Execute()
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext, ContextCallback, object, bool)
   at System.Threading.ExecutionContext.Run(ExecutionContext, ContextCallback, object, bool)
   at System.Threading.Tasks.Task.ExecuteWithThreadLocal(ref Task)
   at System.Threading.Tasks.Task.ExecuteEntry(bool)
   at System.Threading.ThreadPoolWorkQueue.Dispatch()


-----------------------------------------------------------------------------------------------------
Date: 4/24/2023 12:56:22 AM, Tick Count: 83461140 (23:11:01.1400000), Size: 2.30 KB
Process: AeXSvc (4804), Thread ID: 257, Module: Altiris.Database.dll
Priority: 1, Source: AdminDatabaseContext

Entry 2:

DatabaseContext validation failed.
 Context stack: [#0] 257:2370b850-5e2a-4e0c-abb3-ac307009f7b6
 Current: [2/257/-1] {TranOwner,ReadCommitted}, t='Rolledback' ([05fdc407ae8e44649367b0ea91f4dbfa]), s=[#0] 257:2370b850-5e2a-4e0c-abb3-ac307009f7b6, Altiris.NS.ContextManagement.AdminDatabaseContext

Incorrect database context object.
   [InvalidDatabaseContextException]

Exception logged from:
   at Altiris.Diagnostics.Logging.EventLog.ReportException(int, string, string, Exception, string)
   at Altiris.Database.DatabaseContext<>.LogError(string, Exception)
   at Altiris.Database.DatabaseContext<>.ValidateDisposingContext(bool)
   at Altiris.Database.DatabaseContext<>.Dispose(bool)
   at Altiris.Database.DatabaseContext<>.Dispose()
   at Altiris.Database.RetryActionRequest<>.Perform(bool)
   at Altiris.Database.DatabaseContext<>.PerformWithDeadlockRetryHelper(int, int, bool, Getter<IDatabaseContext>, Action<IDatabaseContext>, Action<DeadlockRetryArgs>, bool, string, string)
   at Altiris.NS.SiteWide.SiteTransferRequestHelper.Serve(SiteTransferRequest, out string)
   at Altiris.NS.StandardItems.AgentManagement.Communication.Handlers.AgentActionSiteTransferHandler.Handle(INSAgentConnection, IAgentAction<string>)
   at System.Threading.Tasks.ContinuationResultTaskFromTask<>.InnerInvoke()
   at System.Threading.Tasks.Task.Execute()
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext, ContextCallback, object, bool)
   at System.Threading.ExecutionContext.Run(ExecutionContext, ContextCallback, object, bool)
   at System.Threading.Tasks.Task.ExecuteWithThreadLocal(ref Task)
   at System.Threading.Tasks.Task.ExecuteEntry(bool)
   at System.Threading.ThreadPoolWorkQueue.Dispatch()


-----------------------------------------------------------------------------------------------------
Date: 4/24/2023 12:56:22 AM, Tick Count: 83461140 (23:11:01.1400000), Size: 1.98 KB
Process: AeXSvc (4804), Thread ID: 257, Module: Altiris.Database.dll
Priority: 1, Source: AdminDatabaseContext

Entry 3:

Context dispose failed (dispose)
 Current context: [2/257/-1] {TranOwner,ReadCommitted}, t='Rolledback' ([05fdc407ae8e44649367b0ea91f4dbfa]), s=[#0] 257:2370b850-5e2a-4e0c-abb3-ac307009f7b6, Altiris.NS.ContextManagement.AdminDatabaseContext

Incorrect database context object.
   [InvalidDatabaseContextException @ Altiris.Database.dll]
   at Altiris.Database.DatabaseContext<>.ValidateDisposingContext(bool)
   at Altiris.Database.DatabaseContext<>.Dispose(bool)

Exception logged from:
   at Altiris.Diagnostics.Logging.EventLog.ReportException(int, string, string, Exception, string)
   at Altiris.Database.DatabaseContext<>.LogError(string, Exception)
   at Altiris.Database.DatabaseContext<>.Dispose(bool)
   at Altiris.Database.DatabaseContext<>.Dispose()
   at Altiris.Database.RetryActionRequest<>.Perform(bool)
   at Altiris.Database.DatabaseContext<>.PerformWithDeadlockRetryHelper(int, int, bool, Getter<IDatabaseContext>, Action<IDatabaseContext>, Action<DeadlockRetryArgs>, bool, string, string)
   at Altiris.NS.SiteWide.SiteTransferRequestHelper.Serve(SiteTransferRequest, out string)
   at Altiris.NS.StandardItems.AgentManagement.Communication.Handlers.AgentActionSiteTransferHandler.Handle(INSAgentConnection, IAgentAction<string>)
   at System.Threading.Tasks.ContinuationResultTaskFromTask<>.InnerInvoke()
   at System.Threading.Tasks.Task.Execute()
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext, ContextCallback, object, bool)
   at System.Threading.ExecutionContext.Run(ExecutionContext, ContextCallback, object, bool)
   at System.Threading.Tasks.Task.ExecuteWithThreadLocal(ref Task)
   at System.Threading.Tasks.Task.ExecuteEntry(bool)
   at System.Threading.ThreadPoolWorkQueue.Dispatch()


-----------------------------------------------------------------------------------------------------
Date: 4/24/2023 12:56:22 AM, Tick Count: 83461140 (23:11:01.1400000), Size: 2.01 KB
Process: AeXSvc (4804), Thread ID: 257, Module: Altiris.Database.dll
Priority: 1, Source: AdminDatabaseContext

Entry 4:

Failed to handle action: 'SiteTransfer' (44ae6c80-a798-4bbd-8302-760e5710c0a4), act-id=cc82028b-bc68-46ea-bc84-bbe834493141, plugin=0x0, data=string[254],
 from: agent=f9da896e-aaf1-4e1b-9958-b6b146b684d1, auth=True, addr=132.246.56.244:64036 (Opened), conn-id=ec768ff4-1562-4e38-8879-4c51c8e94158, rage=01:07:03.8491611,
 to: Altiris.NS.StandardItems.AgentManagement.Communication.Handlers.AgentActionSiteTransferHandler,
 total errors: 175 (0 e/s)

Incorrect database context object.
   [InvalidDatabaseContextException @ Altiris.Database.dll]
   at Altiris.Database.DatabaseContext<>.ValidateDisposingContext(bool)
   at Altiris.Database.DatabaseContext<>.Dispose(bool)
   at Altiris.Database.DatabaseContext<>.Dispose()
   at Altiris.Database.RetryActionRequest<>.Perform(bool)
   at Altiris.Database.DatabaseContext<>.PerformWithDeadlockRetryHelper(int, int, bool, Getter<IDatabaseContext>, Action<IDatabaseContext>, Action<DeadlockRetryArgs>, bool, string, string)
   at Altiris.NS.SiteWide.SiteTransferRequestHelper.Serve(SiteTransferRequest, out string)
   at Altiris.NS.StandardItems.AgentManagement.Communication.Handlers.AgentActionSiteTransferHandler.Handle(INSAgentConnection, IAgentAction<string>)
   at System.Threading.Tasks.ContinuationResultTaskFromTask<>.InnerInvoke()
   at System.Threading.Tasks.Task.Execute()
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task)
   at Symantec.AgentActions.Handlers.Symantec.AgentActions.Handlers.AgentActionHandler<,>+<Handle>d__6<,>.MoveNext()

Exception logged from:
   at Altiris.Diagnostics.Logging.EventLog.ReportException(int, string, string, Exception, string)
   at Altiris.NS.AgentManagement.Communication.Handlers.NSAgentActionHandler<>.HandleException(IAgentConnection, IAgentAction, Exception)
   at Symantec.AgentActions.Handlers.Symantec.AgentActions.Handlers.AgentActionHandler<,>+<Handle>d__6<,>.MoveNext()
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext, ContextCallback, object, bool)
   at System.Threading.ExecutionContext.Run(ExecutionContext, ContextCallback, object, bool)
   at System.Runtime.CompilerServices.AsyncMethodBuilderCore+MoveNextRunner.Run()
   at System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction(Action, bool, ref Task)
   at System.Threading.Tasks.Task.FinishContinuations()
   at System.Threading.Tasks.Task.Finish(bool)
   at System.Threading.Tasks.Task.ExecuteWithThreadLocal(ref Task)
   at System.Threading.Tasks.Task.ExecuteEntry(bool)
   at System.Threading.ThreadPoolWorkQueue.Dispatch()


-----------------------------------------------------------------------------------------------------
Date: 4/24/2023 12:56:22 AM, Tick Count: 83461140 (23:11:01.1400000), Size: 2.91 KB
Process: AeXSvc (4804), Thread ID: 257, Module: Altiris.Database.dll
Priority: 2, Source: DatabaseContext.ValidateDisposingContext

Entry 5:

Failed to handle action: 'SiteTransfer' (44ae6c80-a798-4bbd-8302-760e5710c0a4), act-id=cc82028b-bc68-46ea-bc84-bbe834493141, plugin=0x0, data=string[254],
 from: agent=f9da896e-aaf1-4e1b-9958-b6b146b684d1, auth=True, addr=132.246.56.244:64036 (Opened), conn-id=ec768ff4-1562-4e38-8879-4c51c8e94158, rage=01:07:03.8491611,
 total errors: 176 (0 e/s)

One or more errors occurred.
   [AggregateException]

Incorrect database context object.
   [InvalidDatabaseContextException @ Altiris.Database.dll]
   at Altiris.Database.DatabaseContext<>.ValidateDisposingContext(bool)
   at Altiris.Database.DatabaseContext<>.Dispose(bool)
   at Altiris.Database.DatabaseContext<>.Dispose()
   at Altiris.Database.RetryActionRequest<>.Perform(bool)
   at Altiris.Database.DatabaseContext<>.PerformWithDeadlockRetryHelper(int, int, bool, Getter<IDatabaseContext>, Action<IDatabaseContext>, Action<DeadlockRetryArgs>, bool, string, string)
   at Altiris.NS.SiteWide.SiteTransferRequestHelper.Serve(SiteTransferRequest, out string)
   at Altiris.NS.StandardItems.AgentManagement.Communication.Handlers.AgentActionSiteTransferHandler.Handle(INSAgentConnection, IAgentAction<string>)
   at System.Threading.Tasks.ContinuationResultTaskFromTask<>.InnerInvoke()
   at System.Threading.Tasks.Task.Execute()
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task)
   at Symantec.AgentActions.Handlers.Symantec.AgentActions.Handlers.AgentActionHandler<,>+<Handle>d__6<,>.MoveNext()
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at Symantec.AgentActions.Handlers.Symantec.AgentActions.Handlers.AgentActionHandler<,>+<Handle>d__6<,>.MoveNext()
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task)
   at Altiris.NS.AgentManagement.Communication.Connections.NSAgentConnection+<HandleAction>d__36.MoveNext()

Exception logged from:
   at Altiris.NS.Logging.WsNsEventLog.Report(LogType, string, string, Exception)
   at Symantec.AgentActions.Connections.AgentConnection.ReportException(IAgentAction, Exception, LogType)
   at Altiris.NS.AgentManagement.Communication.Connections.NSAgentConnection.HandleException(IAgentAction, Exception)
   at System.Threading.Tasks.ContinuationResultTaskFromTask<>.InnerInvoke()
   at System.Threading.Tasks.Task.Execute()
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext, ContextCallback, object, bool)
   at System.Threading.ExecutionContext.Run(ExecutionContext, ContextCallback, object, bool)
   at System.Threading.Tasks.Task.ExecuteWithThreadLocal(ref Task)
   at System.Threading.Tasks.Task.ExecuteEntry(bool)
   at System.Threading.ThreadPoolWorkQueue.Dispatch()


-----------------------------------------------------------------------------------------------------
Date: 4/24/2023 12:56:22 AM, Tick Count: 83461140 (23:11:01.1400000), Size: 3.10 KB
Process: AeXSvc (4804), Thread ID: 126, Module: Symantec.AgentActions.dll
Priority: 2, Source: AgentConnection

Entry 6:

Transaction being rolled back automatically during dispose. To disable this warning rollback your transaction explicitly!
 context: [1/301/-1] {Tran, TranOwner, ConnOwner, Disposed,ReadCommitted}, t='None' ([fd50da3ee6184e489df0432af5043fcc]), s=[#1] 301:be289414-1880-4fa8-a1a6-61130b2fbbf6, Altiris.NS.ContextManagement.AdminDatabaseContext

Message logged from:
   at Altiris.Diagnostics.Logging.EventLog.ReportException(int, string, string, Exception, string)
   at Altiris.Database.DatabaseContext<>.LogError(int, string, Exception)
   at Altiris.Database.DatabaseContext<>.DisposeOwnTransaction()
   at Altiris.Database.DatabaseContext<>.Dispose(bool)
   at Altiris.Database.DatabaseContext<>.Dispose()
   at Altiris.Database.RetryActionRequest<>.Perform(bool)
   at Altiris.Database.DatabaseContext<>.PerformWithDeadlockRetryHelper(int, int, bool, Getter<IDatabaseContext>, Action<IDatabaseContext>, Action<DeadlockRetryArgs>, bool, string, string)
   at Altiris.NS.SiteWide.SiteTransferRequestHelper.Serve(SiteTransferRequest, out string)
   at Altiris.Web.NS.Agent.SiteTransferRequestHandler.HandleRequest(HttpContext, SiteTransferRequestHandlerData, int)
   at Altiris.NS.WebHandlers.AltirisHttpHandlerBase<>.ProcessRequest(HttpContext, T, int)
   at Altiris.NS.WebHandlers.AltirisHttpHandlerBase<>.ProcessRequest(HttpContext)
   at System.Web.HttpApplication+CallHandlerExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute()
   at System.Web.HttpApplication.ExecuteStepImpl(IExecutionStep)
   at System.Web.HttpApplication.ExecuteStep(IExecutionStep, ref bool)
   at System.Web.HttpApplication+PipelineStepManager.ResumeSteps(Exception)
   at System.Web.HttpApplication.BeginProcessRequestNotification(HttpContext, AsyncCallback)
   at System.Web.HttpRuntime.ProcessRequestNotificationPrivate(IIS7WorkerRequest, HttpContext)
   at System.Web.Hosting.PipelineRuntime.ProcessRequestNotificationHelper(IntPtr, IntPtr, IntPtr, int)
   at System.Web.Hosting.PipelineRuntime.ProcessRequestNotification(IntPtr, IntPtr, IntPtr, int)
   at System.Web.Hosting.UnsafeIISMethods.MgdIndicateCompletion(IntPtr, ref RequestNotificationStatus)
   at System.Web.Hosting.UnsafeIISMethods.MgdIndicateCompletion(IntPtr, ref RequestNotificationStatus)
   at System.Web.Hosting.PipelineRuntime.ProcessRequestNotificationHelper(IntPtr, IntPtr, IntPtr, int)
   at System.Web.Hosting.PipelineRuntime.ProcessRequestNotification(IntPtr, IntPtr, IntPtr, int)

 

HTTP [GET]: https://smpserver.domain.com/altiris/NS/Agent/SiteTransfer.aspx?xml=<resource guid='{B9DA896E-AAF1-4E1B-9958-B6B146B684D1}' op='Done'><site>{5EED8291-AF57-4383-9466-AC4895195316}</site><entity>{54C7A0AE-4FF5-44D4-938A-61ADD3168170}</entity><max>-1</max><total>840004</total><done>840004</done><speed>471</speed></resource>
 ip: [132.246.56.244]; x-sma-version: [8.7.1273.0];
 timings: [[W] 00:31:20.1061507];
 response: [200 OK]; x-smp-nsversion: [8.7.1273.0];
-----------------------------------------------------------------------------------------------------
Date: 4/24/2023 12:56:22 AM, Tick Count: 83461171 (23:11:01.1710000), Size: 3.22 KB
Process: w3wp (5320), Thread ID: 301, Module: Altiris.Database.dll
Priority: 2, Source: AdminDatabaseContext

Environment

ITMS 8.7 RTM (GA)

Cause

Known issue.

Resolution

This issue has been reported and addressed under our ITMS 8.7 RU1 release.

For those in 8.7 RTM version, there is a pointfix under our KB 261271 "CUMULATIVE POST ITMS 8.7 RTM POINT FIXES"  

"SiteTransfer register operations do not commit transactions"

 

 

Powered by Wolken Software