Internal error with Applet or Proxy RDP connection in PAM

Products

• CA Privileged Access Manager (PAM)

Issue/Introduction

Sometimes, connecting to a new Windows system from CA PAM is not possible from either the Java Applet or by using an RDP Proxy with protocol RDP enabled

Typically, attempt at connecting using the Applet will result in message

"The connection was forcibly closed by the remote host"

And connecting by using the RDP proxy will apparently go through the initial stages of the connection but it will finally result in an

"Internal error"

message and the connection will not happen.

However, connecting via the RDP proxy with Protocol "Disabled" works, meaning that this is specifically a problem linked to PAM

Environment

CA PAM all versions

Cause

This is more than likely due to Windows responding to CA PAM with a list of ciphers which is not supported and therefore handshake cannot go through

The list of ciphers supported for PAM as a client in an RDP connection can be found in the documentation:

https://techdocs.broadcom.com/us/en/symantec-security-software/identity-security/privileged-access-manager/4-1-7/configure-policies-to-provision-user-access-to-devices-and-applications/configure-devices/set-up-access-to-a-target-device/access-methods.html

Fundamentally it is required that the Windows server has support for any or all of the following ciphers

TLS_RSA_WITH_AES_256_CBC_SHA256

TLS_DHE_RSA_WITH_AES_128_GCM_SHA256

TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384

The list of ciphers that The RDP server will come up with can be obtained by doing

nmap -sV -p <RDP_port> --script ssl-enum-ciphers <IP_of_target_machine>

For instance

root@<server_name>:~# nmap -sV -p <RDP_port> --script ssl-enum-ciphers <IP_of_target_machine>

Starting Nmap 7.40 ( https://nmap.org ) at 2024-05-28 14:49 UTC

Nmap scan report for <Name_of_target_machine> (<IP_of_target_machine>)
Host is up (0.026s latency).
PORT     STATE SERVICE        VERSION
3389/tcp open  ms-wbt-server?
| ssl-enum-ciphers:
|   TLSv1.2:
|     ciphers:
|       TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (secp384r1) - A
|       TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (rsa 2048) - A
|       TLS_DHE_RSA_WITH_AES_128_GCM_SHA256 (dh 2048) - A
|     compressors:
|       NULL
|     cipher preference: server
|_  least strength: A

Using IIS_Crypto, from Nartac Software or a similar utility may help determine what ciphers are defined in the Windows endpoint that one is trying to connect to.

However, that does not necessarily coincide with the list of ciphers offered in a TLS v1.2 and it may thus be misleading. The best option is to launch a nmap command using the script indicated above and check the result against the list of ciphers supported for RDP client connection

Resolution

The solution consists in enabling in the final Windows endpoint, any or all three of the following algorithms

TLS_RSA_WITH_AES_256_CBC_SHA256

TLS_DHE_RSA_WITH_AES_128_GCM_SHA256

TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384

This can be done manually by accessing

HKEY_LOCAL_MACHINE\System\CurrentControlSet\Control\Cryptography\Configuration\Local\SSL\00010002

and the REG_MULTI_SZ Functions key, and adding/deleting there the ciphers that should be supported. For instance:

Alternatively the IIS_Crypto software mentioned before may be used to graphically enable the desired ciphers. The change will require a reboot.

Additional Information

There is a common misconception in that the TLSv1.2 tab appearing under Cryptography in the Security section under the CA PAM Configuration

corresponds to the ciphers supported for RDP client connection. This is not correct !! Those cipher lists correspond only to the ones used in Web connections as determined by the certificates available in PAM. So this has nothing to do with RDP connections, for which one should refer to the explanations above