Internal error with Applet or Proxy RDP connection in PAM due to cipher mismatch

Products

CA Privileged Access Manager (PAM)

Issue/Introduction

Sometimes, users may experience connection failures when using the RDP Client Applet or RDP Proxy with protocol RDP enabled to access target Windows Servers. This typically occurs because the target server does not support the cipher suites required by the PAM RDP client.

Any attempt at connecting using the Applet will result in message: "The connection was forcibly closed by the remote host"

Furthermore connecting by using the RDP proxy will apparently go through the initial stages of the connection but it will finally result in an "Internal error" message and the connection will not happen.

However, connecting via the RDP proxy with Protocol "Disabled" works, meaning that this is specifically a problem linked to PAM

Environment

Product: CA Privileged Access Manager (PAM)
Versions: 4.2.x, 4.3.0
Target OS: Windows Server 2012 R2, 2016, 2019, 2022 (Standard, Azure Edition)

Cause

The PAM RDP client requires specific TLS 1.2 cipher suites. If these are disabled on the target Windows server the Windows RDP server rejects the connection, because it does not support any of the cipher suites offered by the PAM RDP client.

Resolution

The solution consists in enabling in the final Windows endpoint any or all of the cipher suites supported by the PAM RDP client.

This can be done manually by accessing

HKEY_LOCAL_MACHINE\System\CurrentControlSet\Control\Cryptography\Configuration\Local\SSL\00010002 and the REG_MULTI_SZ Functions key, and adding/deleting there the ciphers that should be supported. For instance:

Alternatively the IIS_Crypto software mentioned before may be used to graphically enable the desired ciphers. The change will require a reboot.

Verify supported ciphers on the target server using #### nmap ####: nmap -sV -p 3389 --script ssl-enum-ciphers <Target_IP>

root@<server_name>:~# nmap -sV -p <RDP_port> --script ssl-enum-ciphers <IP_of_target_machine>

Starting Nmap 7.40 ( https://nmap.org ) at 2024-05-28 14:49 UTC
Nmap scan report for <Name_of_target_machine> (<IP_of_target_machine>)
Host is up (0.026s latency).
PORT STATE SERVICE VERSION
3389/tcp open ms-wbt-server?
| ssl-enum-ciphers:
| TLSv1.2:
| ciphers:
| TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (secp384r1) - A
| TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (rsa 2048) - A
| TLS_DHE_RSA_WITH_AES_128_GCM_SHA256 (dh 2048) - A
| compressors:
| NULL
| cipher preference: server
|_ least strength: A
Open #### gpedit.msc #### on the target server.
Navigate to: Computer Configuration > Administrative Templates > Network > SSL Configuration Settings.
Open SSL Cipher Suite Order and set to Enabled.
Ensure the following PAM-supported ciphers are present:

TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384

- TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
- TLS_DHE_RSA_WITH_AES_256_GCM_SHA384
- TLS_RSA_WITH_AES_256_CBC_SHA256

Reboot the target server to apply changes.

The PAM 4.3 release added several cipher suites to the list. For the built-in RDP client (RDP access methods), the list is documented on page Access Methods. For the RDP Proxy, which establishes the connection to the RDP server when a TCP/UDP service is defined in PAM that launches a native RDP client with application protocol RDP, the list is found on page Create an RDP Proxy Service to Access a Device.

Additional Information

In releases prior to PAM 4.3 the list of ciphers supported by PAM as a client in an RDP connection is as follows:

TLS_RSA_WITH_AES_256_CBC_SHA256

TLS_DHE_RSA_WITH_AES_128_GCM_SHA256

TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384

Note that the first two options are available for connections to RDP servers using an RSA certificate only, while the last option is available for connections to RDP servers using an ECDSA certificate only. If you have an RDP server using an RSA certificate, having the TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384 cipher suite enabled on the server will not help.

The list of ciphers that the RDP server supports can be obtained with an nmap command:

nmap -sV -p <RDP_port> --script ssl-enum-ciphers <IP_of_target_machine> (Described above with the expected output)

OR

Using IIS_Crypto, from Nartac Software or a similar utility may help determine what ciphers are defined in the Windows endpoint that one is trying to connect to.

However, that does not necessarily coincide with the list of ciphers offered in a TLS v1.2 and it may thus be misleading. The best option is to launch an nmap command using the script indicated above and check the result against the list of ciphers supported for RDP client connection

-----------------

Next, there is a common misconception in that the TLSv1.2 tab appearing under Cryptography in the Security section under the CA PAM Configuration

Does this corresponds to the ciphers supported for RDP client connection?

This is not correct! The above cipher lists correspond only to the ones used for incoming HTTPS connections, i.e. this configuration defines what cipher suites PAM accepts when an HTTPS client connects to it, such as when a PAM user logs on to PAM using a browser or the PAM client.