In an overloaded environment, detection process is timing out

book

Article ID: 160136

calendar_today

Updated On:

Products

Data Loss Prevention Network Monitor

Issue/Introduction

In a loaded environment, the log message within the detection chain times out and in the case of Network Monitor will be dropped or, in the case of SMTP Prevent, will be resent by the MTA

The detection_operational_0.log would show the following characteristics.

08/Feb/12:13:47:32:287+0000 [WARNING] (DETECTION.12) Message chain #8 has exceeded the component timeout in Detection Chain. If it hasn't stopped processing in 30 more seconds this process will restart. Working on item RequestProcessor.2, total data length: 0
08/Feb/12:13:48:02:293+0000 [WARNING] (DETECTION.13) Intentionally restarting this process, as Message chain #8 is taking too long processing a message in Detection Chain. Working on item RequestProcessor.2, total data length: 0
08/Feb/12:13:48:02:296+0000 [INFO] (DETECTION.4) Detection is shutting down
08/Feb/12:13:48:07:609+0000 [INFO] (DETECTION.500) Script engine CustomFileScriptEngine initialized
08/Feb/12:13:48:07:611+0000 [INFO] (DETECTION.500) Script engine CustomValidatorScriptEngine initialized
08/Feb/12:13:48:07:625+0000 [INFO] (DETECTION.1) Detection is starting
08/Feb/12:13:48:08:703+0000 [INFO] (DETECTION.8) Detection initializing with the following Channel(s) [Inline SMTP]
08/Feb/12:13:48:08:941+0000 [INFO] (DETECTION.5) Waiting for Detection Server configuration

Resolution

 

The timeout that was hit can be modified when you increase the values in the server settings.  Please note that prior to changing any of these values in a production environment it would be recommended to test them in a dedicated environment.

MessageChain.MaximumComponentTime

The time interval (in milliseconds) allowed before any chain component is restarted.
The maximum time interval (in milliseconds) that a message can remain in a message chain.

ContentExtraction.RunawayTimeout

The time interval (in milliseconds) given to the ContentExtractor to finish processing of any document. If the ContentExtractor does not finish processing some document within this time it will be considered unstable and it will be restarted. This value should be significantly greater than ContentExtraction.LongTimeout.

ContentExtraction.LongTimeout 

The time interval (in milliseconds) given to the ContentExtractor to process a document larger than ContentExtraction.LongContentSize. If the document cannot be processed within the specified time it's reported as unprocessed. This value should be greater than ContentExtraction.ShortTimeout and less than ContentExtraction.RunawayTimeout.