HTTP 500 error when Symantec DLP communicates with Symantec Messaging Gateway (SMG) due to certificate chain requirements

Products

• Messaging Gateway

Issue/Introduction

When Symantec Data Loss Prevention (DLP) attempts to communicate with Symantec Messaging Gateway (SMG) using the FlexResponse API, the operation fails, and you receive an "HTTP 500" error in the logs. The error typically references the DlpQuarantineActionsService, such as:

Message: FlexResponse Action [Email Quarantine Connect Approve Action] failed with message: Input error when reading response from email gateway:java.io.IOException: Server returned HTTP response code: 500 for URL: https://<SMG_HOSTNAME>:8443/brightmail/ws/DlpQuarantineActionsService

This error occurs after the TLS handshake completes successfully but before the SOAP service processes the request, often indicating that the SMG appliance cannot validate the client certificate presented by the DLP Enforce server.

Environment

• Symantec Data Loss Prevention (DLP)
• Symantec Messaging Gateway (SMG) versions prior to 10.9.3

Cause

For SMG versions 10.9.2 and previous, importing only the leaf certificate into the SMG "Applications" certificate store is insufficient for successful validation. The SMG appliance requires the full certificate chain (including root and intermediate CA certificates) to be imported into the Applications tab to establish trust with the DLP Enforce server.

Resolution

Resolution To resolve this issue, ensure the full certificate chain is imported into the SMG appliance:

1. Log in to the SMG Control Center.
2. Navigate to Administration > Certificates > Applications.
3. Ensure the certificate currently used for DLP-to-SMG communication is selected.
4. Import the full certificate chain (the leaf certificate, along with all intermediate and root CA certificates) rather than the leaf certificate alone.
5. Save the changes.
6. Restart the SMG Control Center service to apply the configuration changes.
7. Verify communication by attempting a quarantine release action from the DLP Enforce console.

Note: Certificate handling will be updated in a future release to address this issue.