Understanding the Incident Deletion option "Retain Incident, but Delete Original Message/Attachment(s)/File(s):"

Products

Data Loss Prevention Data Loss Prevention Enforce

Issue/Introduction

After marking incidents with the deletion option of "Retain Incident, but Delete Original Message/Attachment(s)/File(s)"...

1. Why does the Incident Deletion queue count still show 0 preventing manual execution of the incident deletor?

2. Why does it take 2 days before some original messages and attachments get deleted using this option?

3. Why are some Original messages and attachments still present even after several days and several attempts to delete their original message and attachments?

Cause

1. The Incident Deletor queue count is the total number of incidents marked for deletion that still need to be deleted from the database plus the number of original network messages and attachments that have been validated for deletion. The original network messages and attachments are stored in LOB columns in the database or LOB files when externalized storage is used.

2. The LOB deletion validation occurs as part of the incident deletor job after it has deleted incidents and LOBs that were already validated to be deleted. After it runs the first time the queue count will include those LOBs that have been validated for deletion. Thus the job has to run twice for LOB deletions where the incident is to be retained. With a default incident deletor schedule of midnight every day that means two days before selected attachments will be deleted.

Note: The incident deletor can be manually run, but only when the queue has a count greater than 0.

3. LOB data (Original Message and attachments) is only deleted when there are no incidents that have any dependency on that LOB data. Once an incident's attachments are marked for deletion, its dependency on the associated LOB data is removed. Part of the LOB deletion validation is to make sure all incidents that use that LOB data have either been deleted or no longer depend on them.

Resolution

Make sure that the Incident Deletor has executed at least twice since deleting original network message(s) and/or attachment(s)/file(s).

Check the incident's list of "Other policies violated" as these would represent other incidents that are dependent on the same LOB data. All incidents that share the same LOB must all agree that data can be deleted.

Additional Information

Starting our example with 13 incidents.

Incidents 1 - 5 are each a single incident message.
Incidents 6 - 13 are 4 pairs of incidents that share a message.
In this example incidents 6 and 7 share a message and 12 are 13 share a different message.

Say you mark incidents 1 - 10 for attachment only deletion.

The incident deletion queue will show a 0 count because you are not deleting any incidents and no LOB data has been verified as able to be deleted.

Next you mark incident 13 for deletion. Now you have 10 incidents marked for attachment deletion and 1 incident to be deleted. The incident deletion queue will now only show 1 incident because the attachment deletions have not been verified for LOB deletion yet.

Now that the incident deletion queue is greater than 0, you can manually run the incident deletion. When it is finished it will report that 1 incident has been deleted.

The queue will now show a count of 7, all of which are LOB deletions (5 LOBs from the single incident messages and 2 LOBs from the 4 incidents that share 2 messages). Because the count is again greater than 0 it can manually be run. When this second manual deletion finishes it will show that 7 LOBs (attachments) were deleted.

Of the original 13 incidents

Incidents 1 - 9 no longer have their attachments.
Incident 10 while marked to have its attachments deleted, didn't get deleted because incident 11 still uses those attachments.
Incident 13 was deleted but didn't delete the LOB data that both 12 and 13 shared.

Next, select incident 11 to retain the incident but delete attachments and select incident 12 to be deleted.

At this time, the queue count will be 1. Run the deletion job and note that 1 indent was deleted (incident 12) and that the queue is back to a count of 1 again. Run the job again and note that 1 LOB was deleted (from incident 11).

Now incidents 10 and 11 show the attachments have been removed.