Error 'java.lang.OutOfMemoryError at java.io.ByteArrayOutputStream.hugeCapacity' when persisting large incidents over 2GB

Products

Data Loss Prevention Enforce

Issue/Introduction

When persisting incident files larger than 2GB in size, you may encounter the following error in the IncidentPersister_(n).log files:

(SEVERE) Thread: 100 [com.vontu.incidenthandler.message.persist.IncidentThreadPoolExecutor.logProcessingFinished] Working for incident l1609955326001.idc_1609957401069.idc terminated with an exception, leaving in Queue for retry
java.lang.OutOfMemoryError
at java.io.ByteArrayOutputStream.hugeCapacity(ByteArrayOutputStream.java:123)
at java.io.ByteArrayOutputStream.grow(ByteArrayOutputStream.java:117)
at java.io.ByteArrayOutputStream.ensureCapacity(ByteArrayOutputStream.java:93)
at java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:153)
at com.vontu.messaging.chain.message.incident.MessageContent.restoreContent(MessageContent.java:123)
at com.vontu.incidenthandler.message.content.StreamedByteContent.readObject(StreamedByteContent.java:67)
at sun.reflect.GeneratedMethodAccessor233.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1170)
at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2178)
at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2069)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1573)
at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2287)
at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2211)
at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2069)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1573)
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:431)
at java.util.ArrayList.readObject(ArrayList.java:797)
at sun.reflect.GeneratedMethodAccessor1.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1170)
at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2178)
at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2069)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1573)
at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2287)
at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2211)
at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2069)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1573)
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:431)
at com.vontu.incidenthandler.message.persist.IncidentPersistingThread.run(IncidentPersistingThread.java:124)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)

Environment

DLP 15.x

Cause

If a file larger than 2GB (original message) is retained within the incident (.IDC file), the incident will fail to persist due a hard limit on the capacity of Java array objects which is Integer.MAX_VALUE (2,147,483,647).

Note: Increasing the Incident Persister heap will not resolve this particular class of java.lang.OutOfMemory exception.

Resolution

If the large incidents are coming from an Endpoint Discover scan using "Limit Incident Data Retention" Response Rules with the option of "Supported Endpoint Channels -> Retain Original Message" enabled:

Option 1

Remove the Response Rules with Retain Original Message from the policies used in the Endpoint Discover scans.

Option 2

Set a limit of 2000MB or lower in the Endpoint Discover scan targets as shown below so that files larger than 2GB will never be retained: