After an upgrade in place for the test SMP Server (Notification Server or NS) from Windows Server 2012 R2 to 2016 the test site server that is still on 2012 R2 is not able to communicate with the test NS.
ITMS 8.x
We see these errors in the agent logs:
Request 'HTTPS://NotificationServerName.example.local:443/Altiris/NS/Agent/CreateResource.aspx' failed, COM error: An existing connection was forcibly closed by the remote host (0x80072746)
Configure Server Mode: Failed to obtain the machine resource GUID, error: An existing connection was forcibly closed by the remote host (0x80072746)
Failed to register agent. Registration Status 'Not registered'. Next retry in 1 min.
The certificate was missing a Hash.
On the SMP Server, use IISCryto to view the settings.
Click on the "Best Practices" button (above in yellow), then click on the "Apply Button" and reboot the computer.
This fixed the communication issue.
Refer to the following Microsoft documentation for further information about TLS and Schannel settings:
https://learn.microsoft.com/en-us/windows-server/security/tls/tls-registry-settings?tabs=diffie-hellman
https://learn.microsoft.com/en-us/troubleshoot/windows-server/windows-security/restrict-cryptographic-algorithms-protocols-schannel