Win7 and later agents unable to communicate to SMP when using TLS1.2 and certificates that use SHA512 for signature.

Products

• IT Management Suite
• Client Management Suite

Issue/Introduction

Windows 7 and newer unable to communicate to SMP (HTTPS) if TLS other than 1.0 is switched on.

1. Rollout SMA (Symantec Management Agent) to Win7 clients or newer. These client machines will be using HTTPS
2. Switch off TLS 1.0 support
3. Update policies

SMA is getting disconnected after policy update.

Error 1:
Operation 'Head' failed.
Protocol: HTTPS
Host: MySMP.Domain.com:443
Path: /Altiris/NS/Agent/GetClientPolicies.aspx
Http status: 0
Id: {62721E8E-97A0-4890-A3AE-92C82ADD14AC}
Error type: Network error
Error result: 0x80072746
Error code: 0
Error note: SocketIOStrategySyncSelect::Send error
Error message: An existing connection was forcibly closed by the remote host
-----------------------------------------------------------------------------------------------------
Process: AeXNSAgent.exe (2320), Module: AeXNetComms.dll
Priority: 1, Source: NetworkOperation


Error 2:
Policy request failed: An existing connection was forcibly closed by the remote host (0x80072746)
-----------------------------------------------------------------------------------------------------
Process: AeXNSAgent.exe (2320), Module: AeXNSAgent.exe
Priority: 1, Source: ConfigServer

 

Environment

ITMS 7.x, 8.x

Cause

Everything works fine with TLS1.1, the problem is specific to TLS1.2 and certificates that use SHA512 for signature.
 

In case when SHA512+RSA certificate is installed on IIS the server simply resets the connection after receiving the "Client Hello" TLS packet from the agent.
The agent does not know anything about that SHA512+RSA signed certificate, the server log however shows the error:

"An TLS 1.2 connection request was received from a remote client application, but none of the cipher suites supported by the client application are supported
by the server. The SSL connection request has failed."

The root cause description is here:
 

http://blogs.technet.com/b/silvana/archive/2014/03/14/schannel-errors-on-scom-agent.aspx

Resolution

This issue has been reported to our Symantec Development team. Changes have been done (next major release post ITMS 8.0) in the Symantec Management Agent to handle this issue a little bit better.

The current solution is either not use the SHA512+RSA certificate on the server, or enable SHA512+RSA algorithm on the server.

Here is MS KB to enable SHA512+RSA algorithm:
https://support.microsoft.com/en-us/kb/2973337