Unable to establish CEM connection. Error: HTTP status 500: An unexpected condition prevented the server from fulfilling the request (0x8FA101F4)
Article ID: 226921
Updated On:
Products
Issue/Introduction
The client machines in CEM mode are displaying the following error messages when trying to connect to the ITMS environment:
Failed to establish main persistent server connection, error: HTTP status 500: An unexpected condition prevented the server from fulfilling the request (0x8FA101F4)
-----------------------------------------------------------------------------------------------------
Date: 10/19/2021 8:28:42 PM, Tick Count: 2553203 (00:42:33.2030000), Size: 386 B
Process: AeXNSAgent.exe (4508), Thread ID: 7376, Module: AeXNSAgent.exe
Priority: 2, Source: Agent
>>>
[18FC5FD0010, WS: 690, RECV: 3F17F003] UPGRADE request failed, the server might not support WebSocket protocol, error: HTTP status 500: An unexpected condition prevented the server from fulfilling the request (0x8FA101F4)
-----------------------------------------------------------------------------------------------------
Date: 10/19/2021 8:28:42 PM, Tick Count: 2553203 (00:42:33.2030000), Size: 464 B
Process: AeXNSAgent.exe (4508), Thread ID: 19788, Module: AeXNetComms.dll
Priority: 1, Source: SMAIO.WSTransport.Socket
>>>
Policy request failed, COM error: HTTP status 500: An unexpected condition prevented the server from fulfilling the request (0x8FA101F4)
-----------------------------------------------------------------------------------------------------
Date: 10/19/2021 8:28:38 PM, Tick Count: 2549296 (00:42:29.2960000), Size: 366 B
Process: AeXNSAgent.exe (4508), Thread ID: 14312, Module: AeXNSAgent.exe
Priority: 1, Source: ConfigServer
>>>
Operation 'Direct: Head' failed.
Protocol: HTTPS
Host: itmgmt01.example.com:443
Path: /altiris/NS/Agent/GetClientPolicies.aspx
Connection Id: 15.4508
Communication profile Id: {C9673F7F-41C8-4C11-8681-5F76D1B569C6}
Throttling: 0 0 0
Error type: HTTP error
Error code: HTTP status 500: An unexpected condition prevented the server from fulfilling the request (0x8FA101F4)
Error note: 500
Server HTTPS connection info:
Server certificate:
Serial number: xxxxxxxxxx e7 fe 8f 80 ab
Thumbprint: xxxxxxxxxxx 55 65 c9 ab 99 72 a1 a6 ab 63 ce
Cryptographic protocol: TLS 1.2
Cipher suite: TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
Cipher algorithm: AES
Cipher key length: 256
Hash algorithm:
Hash length: 0
Key exchange algorithm: ECDH
Key length: 255
-----------------------------------------------------------------------------------------------------
Date: 10/19/2021 8:28:38 PM, Tick Count: 2549296 (00:42:29.2960000), Size: 1.08 KB
Process: AeXNSAgent.exe (4508), Thread ID: 14312, Module: AeXNetComms.dll
Priority: 1, Source: NetworkOperation
>>>
Calling NS server endpoint 'https://itmgmt01.example.com:443/altiris/NS/Agent/GetClientPolicies.aspx', ID: {32FB5E6D-4E8B-4965-9E4F-9642A99A95E6}
-----------------------------------------------------------------------------------------------------
Date: 10/19/2021 8:28:37 PM, Tick Count: 2547984 (00:42:27.9840000), Size: 378 B
Process: AeXNSAgent.exe (4508), Thread ID: 14312, Module: AeXNSAgent.exe
Priority: 4, Source: Agent
The same certificate was being used for everything: SMP, Gateway, and CEM clients. The expected locations are showing the right certificate assignments (Default Website, Symantec Agent Site, Gateway):
- (Note: for this article itmgmt01.example.com and c5 15 da a5 a7 90 7b 6a bd 55 65 c9 ab 99 72 a1 a6 ab 63 ce as example values)
- The certificate is itmgmt01.example.com with a thumbprint of C5 15 DA A5....
- The default website is using port 443 and itmgmt01.example.com with a thumbprint of C5 15 DA A5..
- Symantec Agent site is using port 4726 and itmgmt01.example.com with a thumbprint of C5 15 DA A5....
- Gateway UI is using port 443 and itmgmt01.example.com with a thumbprint of C5 15 DA A5....
- The Gateway policy has the external name itmgmt01.domain.com with a thumbprint of C5 15 DA A5....
- TLS 1.0, 1.1, 1.2 are enabled in the agent communication profile and Site Server communication profiles.
Confirmed that a static IP address was used for the external IP address for the Internet Gateway.
When the client machine is connected to the internal network it looks fine, both the request configuration and send basic inventory works.
Environment
ITMS 8.x
Cause
There was a misconfiguration and an unusual implementation of how they the external name was use. If the standard CEM implementation steps are followed, the you will know that you need to work on getting a unique external name for your Internet Gateway (something other than the actual SMP server name) that is reachable externally by the client machines, and you will need to generate a new gateway certificate that reflects the external name. Then, update the gateway policy with that new name and add the new thumbprint from the gateway.
Resolution
Use the SMP server name as the external name: itmgmt01.example.com
The gateway policy also had the SMP name because it was the external name to connect to the gateway according to the setup.
The firewall was set up to redirect the external traffic to the gateway but because the client machines were able to resolve the SMP server name directly, they were not trying to connect via the gateway.
Having "itmgmt01.example.com" as the SMP server name, external reachable gateway name, and the name for the primary certificate name, caused some confusion on the client machines in CEM mode.
Since the gateway is using a static IP address (the actual gateway hostname is not reachable externally) that is reachable externally, we suggested the customer change the gateway policy and add the IP address instead of the name for the gateway. And since the client machines are using the same certificate for everything, we didn't have to change the current certificate and thumbprint on the gateway.
Feedback
