Email delivery failed with SMTP ERROR code 501, Fatal error - invalid transmission request, reason - overflow in incoming datastream

Products

• Data Loss Prevention Endpoint Prevent
• Data Loss Prevention Core Package
• Data Loss Prevention Enterprise Suite
• Data Loss Prevention Network Monitor and Prevent for Email
• Data Loss Prevention Network Monitor and Prevent for Email and Web
• Data Loss Prevention Network Prevent for Email

Issue/Introduction

This article describes the resolution for an SMTP 501 error occurring on the DLP Detection Server. The error indicates a violation of SMTP protocol constraints due to a single line in the data stream exceeding the defined character limit.

SMTP_Operational logs.

19/Aug/26:15:45:32:700-0400 [INFO] (SMTP_MESSAGE.5300) Error while processing message (tid=30 cid=Upstream-xxxxxxxxxx message_id=<####@####.com> dlp_id=xxxxxx size=520 sender=####@####.com recipient_count=1 disposition=ERROR code=501 estatus=<5.5.2> text=<Fatal: Invalid transmission request.> rtime=<> dtime=<> mtime=<> reason=<Overflow in incoming datastream.>)

Environment

• Symantec Data Loss Prevention (All versions)
• DLP Detection Server (Network Prevent for Email)

Cause

DLP Detection Server utilizes a configuration setting called RequestProcessor.MaxLineSize, which defaults to 1,048,576 characters. If a message contains a single line of text or encoded data that exceeds this limit, the Detection Server terminates the connection and returns a "501 Fatal: Invalid transmission request" error (reason: "Overflow in incoming datastream"). 

Resolution

Resolution

To resolve this issue, perform one of the following actions:

1. Open the problematic message (or a capture of the message) in a text editor like Notepad++ and navigate to View > Summary to identify if the message contains single lines exceeding 1,048,576 characters. Increase the RequestProcessor.MaxLineSize on the DLP Detection Server to accommodate the required line length:
   • Access the DLP Detection Server advanced server setting.
   • Locate the RequestProcessor.MaxLineSize parameter.
   • Increase the value to a size larger than the longest identified line in the messages (e.g., 3,000,000).
   • Restart the DLP Detection Server services to apply the change.
2. Coordinate with the team managing the sending application to ensure the messages follow standard SMTP guidelines. Update the application export process to insert appropriate line breaks to prevent single-line overflows.

Additional Information

If the issue persists after increasing the character limit, contact Broadcom Support for further investigation. To speak with a customer representative or a Support Engineer, see Contact Support. Scroll to the bottom of the page and click on your respective region.