ALERT: Some images may not load properly within the Knowledge Base Article. If you see a broken image, please right-click and select 'Open image in a new tab'. We apologize for this inconvenience.

Unable to establish CEM connection. Error: HTTP status 500: An unexpected condition prevented the server from fulfilling the request (0x8FA101F4)

book

Article ID: 226921

calendar_today

Updated On:

Products

IT Management Suite

Issue/Introduction

The client machines in CEM mode are displaying the following error messages when trying to connect his ITMS environment:

Failed to establish main persistent server connection, error: HTTP status 500: An unexpected condition prevented the server from fulfilling the request (0x8FA101F4)
-----------------------------------------------------------------------------------------------------
Date: 10/19/2021 8:28:42 PM, Tick Count: 2553203 (00:42:33.2030000), Size: 386 B
Process: AeXNSAgent.exe (4508), Thread ID: 7376, Module: AeXNSAgent.exe
Priority: 2, Source: Agent
>>> 

[18FC5FD0010, WS: 690, RECV: 3F17F003] UPGRADE request failed, the server might not support WebSocket protocol, error: HTTP status 500: An unexpected condition prevented the server from fulfilling the request (0x8FA101F4)
-----------------------------------------------------------------------------------------------------
Date: 10/19/2021 8:28:42 PM, Tick Count: 2553203 (00:42:33.2030000), Size: 464 B
Process: AeXNSAgent.exe (4508), Thread ID: 19788, Module: AeXNetComms.dll
Priority: 1, Source: SMAIO.WSTransport.Socket
>>>

Policy request failed, COM error: HTTP status 500: An unexpected condition prevented the server from fulfilling the request (0x8FA101F4)
-----------------------------------------------------------------------------------------------------
Date: 10/19/2021 8:28:38 PM, Tick Count: 2549296 (00:42:29.2960000), Size: 366 B
Process: AeXNSAgent.exe (4508), Thread ID: 14312, Module: AeXNSAgent.exe
Priority: 1, Source: ConfigServer
>>>

Operation 'Direct: Head' failed. 
Protocol: HTTPS 
Host: itmgmt01.domain.com:443 
Path: /altiris/NS/Agent/GetClientPolicies.aspx 
Connection Id: 15.4508 
Communication profile Id: {C9673F7F-41C8-4C11-8681-5F76D1B569C6} 
Throttling: 0 0 0 
Error type: HTTP error 
Error code: HTTP status 500: An unexpected condition prevented the server from fulfilling the request (0x8FA101F4) 
Error note: 500 
Server HTTPS connection info: 
   Server certificate: 
      Serial number: 5c 2c 00 ed 53 24 cc 89 47 0f 4e e7 fe 8f 80 ab 
      Thumbprint: c5 15 da a5 a7 90 7b 6a bd 55 65 c9 ab 99 72 a1 a6 ab 63 ce 
   Cryptographic protocol: TLS 1.2 
   Cipher suite: TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 
   Cipher algorithm: AES 
   Cipher key length: 256 
   Hash algorithm:  
   Hash length: 0 
   Key exchange algorithm: ECDH 
   Key length: 255
-----------------------------------------------------------------------------------------------------
Date: 10/19/2021 8:28:38 PM, Tick Count: 2549296 (00:42:29.2960000), Size: 1.08 KB
Process: AeXNSAgent.exe (4508), Thread ID: 14312, Module: AeXNetComms.dll
Priority: 1, Source: NetworkOperation
>>>

Calling NS server endpoint 'https://itmgmt01.domain.com:443/altiris/NS/Agent/GetClientPolicies.aspx', ID: {32FB5E6D-4E8B-4965-9E4F-9642A99A95E6}
-----------------------------------------------------------------------------------------------------
Date: 10/19/2021 8:28:37 PM, Tick Count: 2547984 (00:42:27.9840000), Size: 378 B
Process: AeXNSAgent.exe (4508), Thread ID: 14312, Module: AeXNSAgent.exe
Priority: 4, Source: Agent

The customer is using the same certificate for everything: SMP, Gateway, and CEM clients. The expected locations are showing the right certificate assignments (Default Website, Symantec Agent Site, Gateway):
(Note: we will be using itmgmt01.domain.com and c5 15 da a5 a7 90 7b 6a bd 55 65 c9 ab 99 72 a1 a6 ab 63 ce as example values in this article)
The certificate is itmgmt01.domain.com with a thumbprint of C5 15 DA A5....
Default web site is using port 443 and itmgmt01.domain.com with a thumbprint of C5 15 DA A5....
Symantec Agent site is using port 4726 and itmgmt01.domain.com with a thumbprint of C5 15 DA A5....
Gateway UI is using port 443 and itmgmt01.domain.com with a thumbprint of C5 15 DA A5....
The Gateway policy has the external name of itmgmt01.domain.com with a thumbprint of C5 15 DA A5....

TLS 1.0, 1.1, 1.2 are enabled in agent communication profile and Site Server communication profiles. 

The customer has a static IP address for their external IP address for their gateway.

When the client machine is connected into the internal network, it looks fine: request configuration and send basic inventory works.

Cause

It was a misconfiguration from their side and unusual implementation on how they were using their external name. If we follow the standard CEM implementation steps, the customer now knows that he needs to work on getting a unique external name for his gateway (something other than the actual SMP server name) that is reachable externally by the client machines and he needs to generate a new gateway certificate that reflects that external name. Then, update the gateway policy with that new name and add the new thumbprint from the gateway.

Environment

ITMS 8.5, 8.6

Resolution

The customer was using the SMP server name as their external name:  itmgmt01.domain.com
The gateway policy had as well the SMP name because it was the external name to connect to the gateway according to their set up.

Their firewall was set up to redirect the external traffic to the gateway but because the client machines were able to resolve the SMP server name directly, they were not trying to connect via the gateway.

Having "itmgmt01.domain.com" as the SMP server name, external reachable gateway name, and the name for the primary certificate name, caused confusion on the client machines in CEM mode. 

Since the gateway is using a static IP address (the actual gateway hostname is not reachable externally) that is reachable externally, we suggested the customer changing the gateway policy and add the IP address instead of the name for the gateway. And since the client machines are using the same certificate for everything, we didn't have to change the current certificate and thumbprint on the gateway. 

 

Attachments