All Task Servers in the environment are crashing
search cancel

All Task Servers in the environment are crashing

book

Article ID: 164346

calendar_today

Updated On:

Products

Task Server

Issue/Introduction

Scenario 1:

Site servers are crashing for no apparent reason on a mass scale. The following error can be found in the agent logs or Windows event logs:

Fatal error occured in module 'ntdll.dll (6.0.6002.19623)' in 'AtrsHost.exe'.
Process ID: 20704
Thread ID: 21360
Context Record: 000000000FC1D000
Exception Record: 000000000FC1D4D0
Exception Code: C0000005
Flags: 00000000
Address: 0000000077BE842E

RAX=000000010E003CE0 RBX=0000000000000000 RCX=000000010E003CE0 RDX=A5A5A5A5A5A5A5A5 RSI=0000000000000000
RDI=000000010E003CE0 RBP=0000000280037D58 RSP=000000000FC1D598 R8 =0000000000000000 R9 =0000000000000010
R10=000000000000000F R11=000000000FC1D500 R12=000000000E007C90 R13=0000000000000001 R14=0000000000000001
R15=0000000000000000 RIP=0000000077BE842E FLG=00010283
CS=0033 DS=002B SS=002B ES=002B FS=0053 GS=002B

Scenario 2:

The Symantec Management Agent (aka Altiris Agent) is crashing frequently. The agent log shows the following error:

Fatal error occured in module 'ntdll.dll (10.0.14393.4530)' in 'AtrsHost.exe'.
Process ID: 9764
Thread ID: 11980
Context Record: 000000BBC9BFB640
Exception Record: 000000BBC9BFBB30
Exception Code: C0000005
Flags: 00000000
Address: 00007FF9592661F3

RAX=0000000000000000 RBX=000001CE103FAFA8 RCX=000001CE103FAFA8 RDX=0000000000000000 RSI=0000000000000000
RDI=0000000000000000 RBP=000000BBBF702000 RSP=000000BBC9BFBD50 R8 =0000000000000000 R9 =0000000000008000
R10=00007FF9585CFC58 R11=000000BBC9BFAFE0 R12=0000000000000000 R13=000000BBC9BFC1B8 R14=0000000000000001
R15=0000000000000000 RIP=00007FF9592661F3 FLG=00010213
CS=0033 DS=002B SS=002B ES=002B FS=0053 GS=002B

Dump file: C:\ProgramData\Symantec\Symantec Agent\CrashDumps\AtrsHost_9764_11980_B1842ZACS0055_2021-09-17_12-40-20.ez2
-----------------------------------------------------------------------------------------------------
Date: 9/17/2021 9:40:51 AM, Tick Count: 70217312 (19:30:17.3120000), Size: 1.01 KB
Process: AeXAgentUtil.exe (19312), Thread ID: 9624, Module: AeXAgentUtil.exe
Priority: 1, Source: CrashControl

Environment

ITMS 8.x

Cause

Scenario 1:

This was caused by a debug performance counter being set in combination with a bug in Microsoft which caused the ADVAPi32 to crash with an access violation.

Scenario 2:

The crash is in the PPA module and looks like an attempt to use a bad critical section

0:088> .exr -1
ExceptionAddress: 00007ff9592661f3 (ntdll!RtlpWaitOnCriticalSection+0x0000000000000097)
ExceptionCode: c0000005 (Access violation)
ExceptionFlags: 00000000
NumberParameters: 2
Parameter[0]: 0000000000000001
Parameter[1]: 0000000000000024
Attempt to write to address 0000000000000024

 

00 ntdll!RtlpWaitOnCriticalSection
01 ntdll!RtlpEnterCriticalSectionContended
02 ntdll!RtlEnterCriticalSection
03 Common_x64!CReadWriteLock::DecrementReaderCount
04 Common_x64!CReadWriteLock::UnlockReader
05 WMIPlugin_x64!CWbemWrap::GetWbemObject
06 WMIPlugin_x64!WMIPlugin::ProcessGetObjectsByQueryCommand
07 WMIPlugin_x64!WMIPlugin::SendCommands
08 PPA_x64!PPA::Connection::SendCommands
09 PPA_x64!PPA::BaseCommandHandler::HandleSendToProtocolCommands
0a PPA_x64!PPA::GetDataCommandHandler::HandleCommands
0b PPA_x64!PPA::PAL::SendCommand

Resolution

Scenario 1:

Note: This issue should no be present on newer versions of Microsoft OS and Task Server.

Try the following:

  • Remove the following registry value from the site servers:

HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Perflib\Configuration Flags

  • By default this value should not exist and/or be set to 4. Removing this value and restarting the task services should stop the services from crashing

Scenario 2:

  • Remove and reinstall the Altiris Pluggable Protocols Architecture Agent (PPA plug-in).
Powered by Wolken Software