F5 Best Practices for connections between the Enforce Server and the DLP Cloud Service

Products

• Data Loss Prevention Cloud Service for Email
• Data Loss Prevention Cloud Detection Service
• Data Loss Prevention Cloud Detection Service for ICAP
• Data Loss Prevention Cloud Detection Service for REST

Issue/Introduction

The connection between the Enforce Server console and the Cloud Service Gateway (CSG) disconnects every 1-2 minutes when EDM profiles are published to the CSG.

As a result of the disconnects, incidents cannot be delivered to the Enforce Server. Incidents will be queued at the CSG until delivery can be completed.

Environment

Release : 15.x

Component : DLP Enforce, utilizing Cloud Services

Cause

When the Enforce Server replicates EDM profiles to the Cloud Detection Service, the CSG returns a zero window buffer for an extended period of time.

The F5 is configured to reset the connection when zero window buffers are observed for 20 seconds (Overview of the TCP profile (11.x) (f5.com)).

Resolution

Recommendation:

Based on Broadcom’s understanding of the F5, we do not recommend placing the F5 in between the Enforce Server and the CSG for the following reasons

1. The F5’s behavior to reset the connection makes more sense in the context of traditional web servers (i.e. request/response models). However, with respect to DLP, the connection between Enforce Server and the CSG uses a custom protocol developed by Broadcom that was designed to optimize the uploading of policies/profiles and downloading of incidents. DLP multiplexes the TCP connection to be able to upload and download content on the same connection. What this means is that even if the upload is busy (i.e. returns a zero window buffer), the download will continue to occur. Hence F5’s action of resetting the connection due to the extended period of zero windows because it assumes the connection is idle is an incorrect assumption.
2. The nature of how the Enforce Server interacts with the Cloud Detectors vs on-premise Detection Servers is very different. When policies and profiles are replicated to on-premise Detection Servers, the content is delivered within the boundaries of the customer’s on-premise network. Once the on-premise Detection Server receives the data, it will either store it in memory or write the content to the local disk. In the case of the Cloud Service, the content is transmitted over TLS to the Cloud Service Gateway. The CSG will receive the content and perform a series of actions that include segmenting, encrypting, and persisting the data in a secure location from where the Cloud Detectors can load the data for detection. This general process can take time.
3. When the F5 receives zero window buffers, it has to hold this data in memory until it is able to deliver to the destination. It seems F5 recommends against increasing the zero window time because it would mean holding the data in memory for a longer period of time. However, the Enforce Server does not face the same limitations. If the F5 were removed from the network path and the zero window buffers were forwarded directly to Enforce, Enforce would be able to correctly manage the content until the CSG is free to receive more bytes.