RDP Connection Issues with TCP/UDP service using Remmina/FreeRDP Clients and PAM Session Recording from Ubuntu/Linux OS.

Products

• CA Privileged Access Manager (PAM)

Issue/Introduction

Users may experiencing connection errors, specifically SSL HANDSHAKE errors, when attempting to connect to RDP devices via the PAM Client on Ubuntu using Remmina or freeRDP clients, particularly when PAM session recording is enabled. The error encountered is "Cannot connect to RDP server "127.0.0.1" (aka tls_accept_client: SSL_accept() from client failed. -313 unknown error number)."

Environment

PAM 4.x
PAM Client on Linux OS
Device Windows OS
TCP/UDP Service calling remmina (Copylefted libre software) in Linux

Cause

The core issue lies in the behavior of Remmina and wfreerdp clients, which utilize SSL for protocol negotiation even when explicitly configured to use TLS 1.2. PAM's RDP Proxy does not support SSL for security reasons, leading to handshake failures and connection errors.

Technical Explanation from Broadcom Engineering Team:

• SSL Protocol Use by Clients: Remmina and wfreerdp initiate connections using SSL when connecting through the RDP Proxy, despite attempts to enforce TLS 1.2.
  
  
• PAM RDP Proxy Security Policy: PAM's RDP Proxy is designed not to support the SSL protocol due to security vulnerabilities.
  
  
• Working Clients Comparison:
  • The Microsoft RDP client successfully connects through the RDP Proxy because it does not use SSL for protocol negotiation in this specific scenario.
  • Direct connections to an XRDP server using SSL are successful because they bypass the RDP Proxy, thus not encountering PAM's security restriction.
  • The PAM RDP Client (RDP access method) connects to the XRDP with the PAM server only routing the connection, bypassing the PAM RDP Proxy. It also does not use the SSL protocol, only TLS
    
    
• Third-Party Client Limitations: Broadcom has attempted to engage with the freeRDP forum regarding this issue, but their conclusion is that it stems from the SSL library used by these clients, and they have declined further engagement.
  
  
• Unsupported Protocol: Broadcom no longer supports the SSL protocol for security reasons, and many modern SSL libraries (e.g., wolfSSL) have removed SSL support from their products.

Resolution

1. Utilize the PAM RDP Client (RDP access method) : The recommended and working solution is for users to use the PAM RDP Client applet provided with the PAM Client in Ubuntu. 
   
• Note: Clients previously encountering NTLM handshake issues (e.g., "An error occurred in NTLM handshake" / ASN1Exception: security.132) when using the PAM RDP client should ensure their Windows Devices have the "Encryption Oracle Remediation" policy set to "Mitigated." This change has been shown to resolve the NTLM handshake problem and allow the PAM RDP Client applet to connect successfully.
  
  
2. Third-Party Client Support: If the client insists on using Remmina or freeRDP, they will need to engage directly with the respective support teams (Remmina, freeRDP, or Ubuntu) as the root cause lies within their client's SSL implementation and is outside of Broadcom's control and supported protocols.

 

Additional Information

This issue has been thoroughly investigated, and it has been determined that the behavior of Remmina and freeRDP clients in using SSL for RDP proxy connections is fundamentally incompatible with PAM's security posture. Broadcom has exhausted its efforts to resolve this from the PAM side. The recommended solution is to transition to the PAM RDP Client applet.