Unable to connect via CEM mode. Error note: Failed to resolve host name to IP address

Products

IT Management Suite

Issue/Introduction

The customer is trying to connect his client machines using Cloud-enabled Management (CEM). However, when the client machines tries to switch from their internal network to the external one, the client machines fail to make the proper CEM connection.

The Agent log shows a message like this one:

Operation 'CEM: Connect' failed.
Protocol: HTTPS
Original host: SMP01.domain.com:443
Real host: gatewayalt03:443
Path: /
Connection id: 50.5772
Communication profile id: {CE93F603-DE80-4753-966E-566F747ED5CC}
Throttling: 0 0 0
Error type: DNS error
Error code: No such host is known (11001)
Error note: Failed to resolve host name to IP address
-----------------------------------------------------------------------------------------------------
Date: 5/14/2020 9:41:17 AM, Tick Count: 41363468 (11:29:23.4680000), Size: 609 B
Process: AeXNSAgent.exe (5772), Thread ID: 5168, Module: AeXNetComms.dll
Priority: 1, Source: NetworkOperation

Environment

ITMS 8.x

Cause

The customer was using the internal name of his gateway rather than the external one. In this example, the Internet Gateway internal name was "gatewayalt03", however, the external one was "gatewayext.domain.com".
The client machine was not able to resolved the internal name from its external connection. That is why the agent log message referred to:
Error type: DNS error
Error code: No such host is known (11001)
Error note: Failed to resolve host name to IP address

If we enable verbose logging on the agent logs, you should see also some entries like these:

Entry 1:
Operation 'Direct: Connect' failed.
Protocol: HTTPS
Host: SMP01.domain.com:443
Path: /
Connection Id: 50.5772
Communication profile Id: {CE93F603-DE80-4753-966E-566F747ED5CC}
Throttling: 0 0 0
Error type: DNS error
Error code: No such host is known (11001)
Error note: Failed to resolve FQDN and short host name to IP address
-----------------------------------------------------------------------------------------------------
Date: 5/14/2020 9:41:14 AM, Tick Count: 41360125 (11:29:20.1250000), Size: 592 B
Process: AeXNSAgent.exe (5772), Thread ID: 5168, Module: AeXNetComms.dll
Priority: 8, Source: NetworkOperation

Entry 2:
Unable to resolve a host name gatewayalt03 to IP address, error: No such host is known (11001)
-----------------------------------------------------------------------------------------------------
Date: 5/14/2020 9:41:17 AM, Tick Count: 41363468 (11:29:23.4680000), Size: 331 B
Process: AeXNSAgent.exe (5772), Thread ID: 5168, Module: AeXNetComms.dll
Priority: 8, Source: HttpConnection

if you use Microsoft "PSPing" tool (https://docs.microsoft.com/en-us/sysinternals/downloads/psping), you can try and see if from your client machine you can have access to your gateway.
If you run PSPing.exe from the command prompt on your client machine, you should see something like this where your gateway name is not responding with the appropriate IP references if the internal name is used:

C:\Desktop\PSTools>psping.exe YOURINTERNALGATEWAYname.domain.com:443

PsPing v2.01 - PsPing - ping, latency, bandwidth measurement utility
Copyright (C) 2012-2014 Mark Russinovich
Sysinternals - www.sysinternals.com

TCP connect to 1.1.1.1:443:
5 iterations (warmup 1) ping test:
Connecting to 1.1.1.1:443: (warmup): from 0.0.0.0:60072:
The remote computer refused the network connection.
Connecting to 1.1.1.1:443:: from 0.0.0.0:60073:
The remote computer refused the network connection.
Connecting to 1.1.1.1:443:: from 0.0.0.0:60074:
The remote computer refused the network connection.
Connecting to 1.1.1.1:443:: from 0.0.0.0:60075:
The remote computer refused the network connection.
Connecting to 1.1.1.1:443:: from 0.0.0.0:60076:
The remote computer refused the network connection.

TCP connect statistics for 200.155.109.78:443:
Sent = 4, Received = 0, Lost = 4 (100% loss),
Minimum = 0.00ms, Maximum = 0.00ms, Average = 0.00ms

Now, if you use the proper external Internet Gateway name and it is open to external communications, you should see something like this while using PSPing:

C:\Desktop\PSTools>psping.exe YOUREXTERNALGATEWAYname.domain.com:443

PsPing v2.01 - PsPing - ping, latency, bandwidth measurement utility
Copyright (C) 2012-2014 Mark Russinovich
Sysinternals - www.sysinternals.com

TCP connect to 1.1.1.1:443::
5 iterations (warmup 1) connecting test:
Connecting to 1.1.1.1:443: (warmup): 79.11ms
Connecting to 1.1.1.1:443:: 79.48ms
Connecting to 1.1.1.1:443:: 80.64ms
Connecting to 1.1.1.1:443:: 81.26ms
Connecting to 1.1.1.1:443:: 81.11ms

TCP connect statistics for 52.40.239.220:443:
Sent = 4, Received = 4, Lost = 0 (0% loss),
Minimum = 79.48ms, Maximum = 81.26ms, Average = 80.62ms

You can also use Powershell Command Test-NetConnection <ipaddress> -port <port> to get similar results to PSPing:

ComputerName : ServerName
RemoteAddress : 1.1.1.1
RemotePort : 443
InterfaceAlias : tap06c3eff7-b0
SourceAddress : 1.1.1.2
TcpTestSucceeded : True

More information on Test-NetConnection: https://docs.microsoft.com/en-us/powershell/module/nettcpip/test-netconnection?view=windowsserver2022-ps

Resolution

For the purpose of this article, we will use the troubleshooting steps that we followed. So in that way, you can validate in which area the issue may be occurring.
We will use the following names as examples:
SMP Name: SMP01.domain.com:443
Internal Gateway name: gatewayalt03
External Gateway name: gatewayext.domain.com

Go to the Internet Gateway and check its configuration. Open the Internet Gateway UI. Check the certificate in use. The customer was using a certificate called "SMP01 CEM" with thumbprint "08 95 dc 19..."
Go to the SMP Server. Open IIS Manager. Check the Default Web Site binding for port 443. The Default Web Site was using a certificate called "SMP01" with thumbprint "b9 2e 62 29..."
Check the Symantec Agent Site and its binding as well in IIS Manager. The Symantec Agent Site was using a certificate called "Gatewayalt03" with thumbprint "08 95 dc 19..."
Go to the SMP Console. Go to Settings>Notification Server>Cloud-enable Management. Check the "Cloud-enabled Management Settings" policy and notice the name of the gateway is listed. In this example, it appeared as "GATEWAYALT03" with thumbprint of "08 95 dc 19..."
In the same Cloud-enabled Management tree, Go to the "Cloud-enabled Management Agent IIS Website Settings". In this example, it had the FQDN as "Altiris02" instead of the actual SMP name of "SMP01".
Since they shouldn't be using the gateway certificate for the Symantec Agent Site (because the certificate should match the SMP name so the client machines can actually talk back to it), we decided to use the certificate that the Default Website is already using (only if the original CEM certificate is no longer available).
We also had to change the "FQDN" and "Certificate" registry under "HKLM\Software\Altiris\NS Agent Site" to match the SMP name on the certificate and the thumbprint of it.
After that, we created a CEM package and installed it on that client machine. Still failed to connect via CEM.
We checked the agent logs and we saw this again:
Operation 'CEM: Connect' failed.
Protocol: HTTPS
Original host: SMP01.domain.com:443
Real host: gatewayalt03:443
Path: /
Connection id: 50.5772
Communication profile id: {CE93F603-DE80-4753-966E-566F747ED5CC}
Throttling: 0 0 0
Error type: DNS error
Error code: No such host is known (11001)
Error note: Failed to resolve host name to IP address

and

Unable to resolve a host name gatewayalt03 to IP address, error: No such host is known (11001)

gatewayalt03 is the physical name of the gateway but the Symantec management agent should be actually using its external name: GATEWAYEXT.DOMAIN.COM.
So we went back to the "Cloud-enabled Management Settings" policy and changed the name of the gateway that was listed as "GATEWAYALT03" to "GATEWAYEXT.DOMAIN.COM".
Then we created a new CEM package and try on ones of those CEM Client machines that were not able to connect internally to get the changes done. It worked.