Error: "The request requires user authentication (0x8FA10191)" when clients not able to register to a Task Server

Products

Client Management Suite IT Management Suite IT Management Suite

Issue/Introduction

After changing the Application Identity to a new account and password, the Notification Server, some other Task Servers, and many clients were not able to register to a Task Server.

Entries seen in the logs are:

Entry 1:

Operation 'Direct: Get' failed.
Protocol: HTTPS
Host: smpserver.example.com:443
Path: /altiris/TaskManagement/ClientTask/GetClientData.aspx
Connection Id: 0.2096
Communication profile Id: {GUID}
Throttling: 0 0 0
Error type: HTTP error
Error code: HTTP status 401: The request requires user authentication (0x8FA10191)
Error note: Authentication failed, server refused to authenticate with the provided credentials or the user account is locked
Server HTTPS connection info:
Server certificate:
Serial number: [serial number]
Thumbprint: [thumbprint]
Cryptographic protocol: TLS 1.2
Cipher suite: TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
Cipher algorithm: AES
Cipher key length: 256
Hash algorithm:
Hash length: 0
Key exchange algorithm: ECDH
Key length: 255
-----------------------------------------------------------------------------------------------------
Date: 8/6/2024 7:17:33 AM, Tick Count: 281777187 (3.06:16:17.1870000), Size: 1.16 KB
Process: AtrsHost.exe (2096), Thread ID: 4768, Module: AeXNetComms.dll
Priority: 1, Source: NetworkOperation

Entry 2:

Web request 'GET' failed: https://smpserver.example.com:443/altiris/TaskManagement/ClientTask/GetClientData.aspx?clientTaskServerGuid=guidhere&resourceGuid=guidhere. Result code: 401 Unauthorized.
[Altiris.DotNetLib.Interop.AeXNetComms.Web.Exceptions.AeXWebRequestException @ Altiris.DotNetLib]
at Altiris.DotNetLib.Interop.AeXNetComms.Web.AeXWebRequest.Execute(String requestUriString, Guid cnnProfileId, TimeSpan timeout, Boolean isPost, String data)
at Altiris.DotNetLib.Helpers.AtrsHttpOps.Execute(String url, Guid cnnProfileId, TimeSpan timeout, Boolean isPost, String data)
at Altiris.ClientTask.Server.Communication.NotificationServerWebConnection.PostToNotificationServerWithRetry(String url, Guid cnnProfileId, String data, Int32 nMaxAttempts, TimeSpan timeout)

One or more errors occurred.
[System.AggregateException @ mscorlib]
at System.Threading.Tasks.Task`1.GetResultCore(Boolean waitCompletionNotification)
at Altiris.DotNetLib.Interop.AeXNetComms.AeXAsyncRequest.Execute()
at Altiris.DotNetLib.Interop.AeXNetComms.Web.AeXWebRequest.Execute(String requestUriString, Guid cnnProfileId, TimeSpan timeout, Boolean isPost, String data)

Web request 'GET' failed: https://smpserver.example.com:443/altiris/TaskManagement/ClientTask/GetClientData.aspx?clientTaskServerGuid=guidhere&resourceGuid=guidhere. Result code: 401 Unauthorized.
[Altiris.DotNetLib.Interop.AeXNetComms.Web.Exceptions.AeXWebRequestException @ Altiris.DotNetLib]
at Altiris.DotNetLib.Interop.AeXNetComms.Web.AeXWebRequest.OperationCallback(IAeXNetworkTransport5 transport)
at Altiris.DotNetLib.Interop.AeXNetComms.AeXRequest.ExecuteRequestSyncCallbackImpl(IAeXNetworkTransport5 transport)
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Altiris.DotNetLib.Interop.AeXNetComms.AeXAsyncRequest.<ExecuteCallback>d__56.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Altiris.DotNetLib.Interop.AeXNetComms.AeXTransportInstance.<ExecuteRequestCallback>d__13.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Altiris.DotNetLib.Interop.AeXNetComms.AeXPooledTransportInstance.<ExecuteRequestCallback>d__8.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Altiris.DotNetLib.Interop.AeXNetComms.AeXTransportManager.<ExecuteRequest>d__44.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Altiris.DotNetLib.Interop.AeXNetComms.AeXAsyncRequest.<ExecuteAsync>d__55.MoveNext()

HTTP error occurred
[System.Runtime.InteropServices.COMException @ Altiris.AeXNetwork.Transport.1]
at Interop.AeXNetComms.AeXNetworkTransportClass.IAeXNetworkTransport5_Get(String url, Int32 decompress)
at Altiris.DotNetLib.Interop.AeXNetComms.Web.AeXWebRequest.OperationCallback(IAeXNetworkTransport5 transport)

COM Exception errcode: 0x80042D21

Exception logged from:
at Altiris.DotNetLib.Logging.AtrsLog.ExceptionMessage(String message, Exception exception)
at Altiris.ClientTask.Server.Logging.NSAgentLog.ReportMessage(Severity severity, String moduleName, String source, Exception exception, String message, Object[] arguments)
at Altiris.ClientTask.Server.Communication.NotificationServerWebConnection.PostToNotificationServerWithRetry(String url, Guid cnnProfileId, String data, Int32 nMaxAttempts, TimeSpan timeout)
at Altiris.ClientTask.Server.Communication.NotificationServerWebConnection.PostToNotificationServer(String url, Guid cnnProfileId, String postData, String& respData, Boolean& dataEncryped, Int32 nMaxAttempts, TimeSpan timeout)
at Altiris.ClientTask.Server.Communication.NotificationServerWebConnection.Altiris.TaskManagement.Common.ClientTask.Communication.IInternalNotificationServerConnection.GetClientData(Guid resourceGuid)
at Altiris.ClientTask.Server.ClientTaskServer.RegisterClient(AltirisResourceGuid resourceGuid, IPAddress ipAddress, Boolean bLastResort, Boolean bBeingActive, String configurationXml, String crc, Hashtable& output, IAuthAgentConnection wsConnection, Version version, Nullable`1 resTypeGuid, String systemType)
at Altiris.ClientTask.Server.ClientTaskServer.ProcessRemoteRegisterClient(Hashtable input, Hashtable& output)
at Altiris.DotNetLib.RemotingRequestService.Altiris.DotNetLib.IRemotingRequestService.HandleRequest(String service, Hashtable parameters)
at SyncInvokeHandleRequest(Object , Object[] , Object[] )
at System.ServiceModel.Dispatcher.SyncMethodInvoker.Invoke(Object instance, Object[] inputs, Object[]& outputs)
at System.ServiceModel.Dispatcher.DispatchOperationRuntime.InvokeBegin(MessageRpc& rpc)
at System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessage5(MessageRpc& rpc)
at System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessage11(MessageRpc& rpc)
at System.ServiceModel.Dispatcher.MessageRpc.Process(Boolean isOperationContextSet)
at System.ServiceModel.Dispatcher.ChannelHandler.DispatchAndReleasePump(RequestContext request, Boolean cleanThread, OperationContext currentOperationContext)
at System.ServiceModel.Dispatcher.ChannelHandler.HandleRequest(RequestContext request, OperationContext currentOperationContext)
at System.ServiceModel.Dispatcher.ChannelHandler.AsyncMessagePump(IAsyncResult result)
at System.ServiceModel.Dispatcher.ChannelHandler.OnAsyncReceiveComplete(IAsyncResult result)
at System.Runtime.Fx.AsyncThunk.UnhandledExceptionFrame(IAsyncResult result)
at System.Runtime.AsyncResult.Complete(Boolean completedSynchronously)
at System.ServiceModel.Channels.TransportDuplexSessionChannel.TryReceiveAsyncResult.OnReceive(IAsyncResult result)
at System.Runtime.Fx.AsyncThunk.UnhandledExceptionFrame(IAsyncResult result)
at System.Runtime.AsyncResult.Complete(Boolean completedSynchronously)
at System.ServiceModel.Channels.SynchronizedMessageSource.ReceiveAsyncResult.OnReceiveComplete(Object state)
at System.ServiceModel.Channels.SessionConnectionReader.OnAsyncReadComplete(Object state)
at System.Runtime.Fx.AsyncThunk.UnhandledExceptionFrame(IAsyncResult result)
at System.Net.LazyAsyncResult.Complete(IntPtr userToken)
at System.Net.LazyAsyncResult.ProtectedInvokeCallback(Object result, IntPtr userToken)
at System.Net.Security.NegotiateStream.ProcessFrameBody(Int32 readBytes, Byte[] buffer, Int32 offset, Int32 count, AsyncProtocolRequest asyncRequest)
at System.Net.Security.NegotiateStream.ReadCallback(AsyncProtocolRequest asyncRequest)
at System.Net.AsyncProtocolRequest.CompleteRequest(Int32 result)
at System.Net.FixedSizeReader.CheckCompletionBeforeNextRead(Int32 bytes)
at System.Net.FixedSizeReader.ReadCallback(IAsyncResult transportResult)
at System.Runtime.AsyncResult.Complete(Boolean completedSynchronously)
at System.ServiceModel.Channels.ConnectionStream.IOAsyncResult.OnAsyncIOComplete(Object state)
at System.ServiceModel.Channels.PipeConnection.OnAsyncReadComplete(Boolean haveResult, Int32 error, Int32 numBytes)
at System.ServiceModel.Channels.OverlappedContext.CompleteCallback(UInt32 error, UInt32 numBytes, NativeOverlapped* nativeOverlapped)
at System.Runtime.Fx.IOCompletionThunk.UnhandledExceptionFrame(UInt32 error, UInt32 bytesRead, NativeOverlapped* nativeOverlapped)
at System.Threading._IOCompletionCallback.PerformIOCompletionCallback(UInt32 errorCode, UInt32 numBytes, NativeOverlapped* pOVERLAP)

-----------------------------------------------------------------------------------------------------
Date: 8/6/2024 7:17:33 AM, Tick Count: 281777187 (3.06:16:17.1870000), Size: 8.56 KB
Process: AtrsHost.exe (2096), Thread ID: 4768, Module: AtrsHost.exe
Priority: 1, Source: NotificationServerWebConnection

Environment

ITMS 8.x

Cause

The Agent Connectivity Credential in the Notification Server communications profile was configured to use the old Application Identity account and password.

Resolution

Disabled the override of the Global Agent Settings policy > Authentication tab:

Under Settings>All Settings>Agents/Plug-ins>Symantec Management Agent>Symantec Management Agent Communication Profiles.
Select your agent communication profile and under "The Agent Connectivity Credentials that are defined on the 'Global Agent Settings' page are selected", click "Edit" and see if you have an account manually typed here. If so, re-type the desired account or just select "use application credentials")

Also, check under Settings>All Settings>Agents/Plug-ins>Symantec Management Agent>Settings>Global Agent Settings>Authentication tab, if you have a specific account. If you do, verify that it is a valid one with the correct username and password.

Note: You can also change it to "Use application credentials" and see if the issue still persists.

Additional Information

In some rare cases you could also try the following:

Update your ACC account, either by adding a new one or retyping the password so this forces the systems to update its reference for this account.
On the SMP Server, under Default Web Site/altiris/TaskManagement/ClientTask virtual directory, enable "Anonymous Authentication" under "Authentication" section, so any Site Server trying to reach the registration process and still needed the updated password for ACC account, could actually register.
And, for the Task Server that is not working for its task server registration, do the same but on that Task Server under IIS Manager / Default Web Site/altiris/ClientTaskServer virtual directory, enable "Anonymous Authentication" under "Authentication" section, so it could register to itself.
After you are able to validate that Task Server registration is back to normal and have given enough time (usually just 24hrs is enough) to get the ACC account updated on your Site Servers, you can revert back the changes done for "Anonymous Authentication" under "Authentication" section in IIS Manager.