Internal error with Applet or Proxy RDP connection in PAM

Products

CA Privileged Access Manager (PAM)

Issue/Introduction

Sometimes, connecting to a new Windows system from CA PAM is not possible from either the Java Applet or by using an RDP Proxy with protocol RDP enabled

Typically, attempt at connecting using the Applet will result in message

"The connection was forcibly closed by the remote host"

And connecting by using the RDP proxy will apparently go through the initial stages of the connection but it will finally result in an

"Internal error"

message and the connection will not happen.

However, connecting via the RDP proxy with Protocol "Disabled" works, meaning that this is specifically a problem linked to PAM

Cause

This is more than likely due to the Windows RDP server rejecting the connection, because it does not support any of the cipher suites offered by the PAM RDP client.

In releases prior to PAM 4.3 the list of ciphers supported by PAM as a client in an RDP connection is as follows:

TLS_RSA_WITH_AES_256_CBC_SHA256

TLS_DHE_RSA_WITH_AES_128_GCM_SHA256

TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384

Note that the first two options are available for connections to RDP servers using an RSA certificate only, while the last option is available for connections to RDP servers using an ECDSA certificate only. If you have an RDP server using an RSA certificate, having the TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384 cipher suite enabled on the server will not help.

The list of ciphers that the RDP server supports can be obtained with an nmap command:

nmap -sV -p <RDP_port> --script ssl-enum-ciphers <IP_of_target_machine>

For instance

root@<server_name>:~# nmap -sV -p <RDP_port> --script ssl-enum-ciphers <IP_of_target_machine>

Starting Nmap 7.40 ( https://nmap.org ) at 2024-05-28 14:49 UTC

Using IIS_Crypto, from Nartac Software or a similar utility may help determine what ciphers are defined in the Windows endpoint that one is trying to connect to.

However, that does not necessarily coincide with the list of ciphers offered in a TLS v1.2 and it may thus be misleading. The best option is to launch an nmap command using the script indicated above and check the result against the list of ciphers supported for RDP client connection

Resolution

The solution consists in enabling in the final Windows endpoint any or all of the cipher suites supported by the PAM RDP client.

This can be done manually by accessing

HKEY_LOCAL_MACHINE\System\CurrentControlSet\Control\Cryptography\Configuration\Local\SSL\00010002

and the REG_MULTI_SZ Functions key, and adding/deleting there the ciphers that should be supported. For instance:

Alternatively the IIS_Crypto software mentioned before may be used to graphically enable the desired ciphers. The change will require a reboot.

The PAM 4.3 release added several cipher suites to the list. For the built-in RDP client (RDP access methods), the list is documented on page Access Methods. For the RDP Proxy, which establishes the connection to the RDP server when a TCP/UDP service is defined in PAM that launches a native RDP client with application protocol RDP, the list is found on page Create an RDP Proxy Service to Access a Device.

Additional Information

There is a common misconception in that the TLSv1.2 tab appearing under Cryptography in the Security section under the CA PAM Configuration

corresponds to the ciphers supported for RDP client connection. This is not correct! Those cipher lists correspond only to the ones used for incoming HTTPS connections, i.e. this configuration defines what cipher suites PAM accepts when an HTTPS client connects to it, such as when a PAM user logs on to PAM using a browser or the PAM client.