How to troubleshoot Error code 1504 - Downstream TLS Handshake Failed
search cancel

How to troubleshoot Error code 1504 - Downstream TLS Handshake Failed

book

Article ID: 162309

calendar_today

Updated On:

Products

Data Loss Prevention Network Prevent for Email Data Loss Prevention

Issue/Introduction

You have configured TLS communication between Email prevent and downstream MTA. However, there is a message that the Downstream "TLS Handshake Failed".
NOTE: This is related to All versions of DLP received following error on Enforce Server console.

Error Code 1504
Downstream TLS Handshake Failed
TLS handshake with downstream MTA smtp-outbound.abcd.com/10.xxx.xxx.xxx:25 failed. Please check the SmtpPrevent and RequestProcessor logs for more information.


SmtpPrevent_Operational0.log shows following error:


                 -----------------------------------++++++++++++++++++++++++++++++++++++++++---------------------------------------
09/Jun/15:10:12:06:885-0500 [INFO] (SMTP_CONNECTION.1201) Connection accepted (tid=2c cid=1 local=10.xxx.xxx.xxx:25 remote=10.xxx.xxx.xxx:27829)
09/Jun/15:10:12:07:057-0500 [INFO] (SMTP_CONNECTION.1203) Forward connection established (tid=2c cid=2 local=10.xxx.xxx.xxx:1499 remote=205.xxx.xxx.xxx:25)
09/Jun/15:10:12:07:479-0500 [INFO] (SMTP_CONNECTION.5209) TLS handshake completed (tid=2c cid=1 local=10.xxx.xxx.xxx:25 remote=10.xxx.xxx.xxx:27829 peer=<unverified> protocol=<TLSv1> cipher=<TLS_RSA_WITH_AES_128_CBC_SHA>)
09/Jun/15:10:12:07:635-0500 [SEVERE] (SMTP_CONNECTION.5208) TLS handshake failed (tid=2c cid=2 local=10.xxx.xxx.xxx:1499 remote=205.xxx.xxx.xxx:25 reason=General SSLEngine problem)
09/Jun/15:10:12:07:635-0500 [INFO] (SMTP_CONNECTION.5203) Forward connection error (tid=2c cid=2 mta=205.xxx.xxx.xxx:25 reason=General SSLEngine problem)
09/Jun/15:10:12:07:635-0500 [SEVERE] (SMTP_CONNECTION.5204) Peer disconnected unexpectedly (tid=2c cid=2 local=10.xxx.xxx.xxx:1499 remote=205.xxx.xxx.xxx:25 reason=General SSLEngine problem)
09/Jun/15:10:12:07:651-0500 [INFO] (SMTP_CONNECTION.1205) Service connection closed (tid=2c cid=1 local=10.xxx.xxx.xxx:25 remote=10.xxx.xxx.xxx:27829 messages=0 time=0.77s)
                -----------------------------------++++++++++++++++++++++++++++++++++++++++---------------------------------------
 

Cause

This issue occurs mostly because the downstream MTA fails to pass Symantec DLP Email Prevent authentication.

Resolution

Start troubleshooting by validating certificate which Email Prevent uses to establish secure communication with downstream MTA. 
Follow below steps to do certificate validation.

1. Get downsteam MTA certificate (public key) using openssl tool and match with certificate you have imported into Email Prevent.

2. To get downstream MTA certificate (public key) you need to download OpenSSL windows binary distributions  from  http://openssl.org
The distribution consists of two separate binaries

  • Recent Visual C Redistributables (You will need to install latest redistributables first to ensure proper functioning of OpenSSL)
  • OpenSSL Installer

3. To connect to the downstream MTA using openssl issue the s_client command  in openSSL and connect to SMTP Prevent using STARTTLS and SMTP
The response appears like:

c:\OpenSSL-Win64\bin>openssl s_client -connect <your-SMTP-server.company.com>:25 -starttls smtp -crlf

NOTE: output of this command looks like one given in the file "output.txt" attached to this KB.

4. Output will show Server certificate which you need to match with certificate you have imported in Email Prevent.

5. Format which this shows is normally pem and file type is *.cer.

6. If in case you see mismatch then contact to messaging team or SMTP vendor for clarification.

7. If in case you find both certificate matches then open case with support for further troubleshooting.

Additional Information

As per our Help Center topic: About TLS authentication (broadcom.com)

"When TLS is requested, each successive proxy in the email chain must authenticate itself to the previous server to establish an end-to-end TLS connection. Successful authentication requires that each mail server stores a valid certificate for the next-hop mail server in its trust store. For example, Network Prevent for Email Server must authenticate itself to the sending MTA, and the downstream MTA or hosted email service must authenticate itself to Network Prevent for Email Server."

Attachments

output.txt get_app