Detection stops working for some Data Identifiers

Products

Data Loss Prevention Enforce Data Loss Prevention Cloud Detection Service Data Loss Prevention Cloud Service for Email Data Loss Prevention

Issue/Introduction

You have noticed that detection for certain Data Identifiers stops even after working previously.

A Data Identifier ("DI") is one of DLP's proprietary expressions which are used to detect content with identifiable patterns, e.g., US Social Security Numbers, Credit Card Numbers, etc.

To improve efficiency and reduce false positives in detection, DIs have additional configuration options called Validators, which run additional checks on data as part of analysis.

There are errors in the logs indicating problems with DI Validators, such as these found in the Enforce Server MonitorController0.log:

May 8, 2021 11:46:53 PM com.symantec.dlp.services.task.AllItemsUpdateTask run
WARNING: An exception has been thrown while updating the cache DataIdentifierValidatorObjReference
org.springframework.orm.ObjectRetrievalFailureException: Object [id=141] was not of the specified subclass 
[com.vontu.enforceentities.policy.CustomScriptDiValidator] : loaded object was of wrong class class com.vontu.enforceentities.policy.SystemDiValidator; 
nested exception is org.hibernate.WrongClassException: Object [id=141] was not of the specified subclass [com.vontu.enforceentities.policy.CustomScriptDiValidator]
 : loaded object was of wrong class class com.vontu.enforceentities.policy.SystemDiValidator

In most cases, this error prevents the references from loading:

May 8, 2021 11:46:55 PM com.symantec.dlp.services.policy.task.PolicyPublisherTask addItemsToShippingPackage
WARNING: Content for object type DataIdentifierValidatorObjReference with id 38 not found.
May 8, 2021 11:46:55 PM com.symantec.dlp.services.policy.task.PolicyPublisherTask addItemsToShippingPackage
WARNING: Content for object type DataIdentifierValidatorObjReference with id 26 not found.
May 8, 2021 11:46:55 PM com.symantec.dlp.services.policy.task.PolicyPublisherTask addItemsToShippingPackage
WARNING: Content for object type DataIdentifierValidatorObjReference with id 13 not found.
May 8, 2021 11:46:55 PM com.symantec.dlp.services.policy.task.PolicyPublisherTask addItemsToShippingPackage
WARNING: Content for object type DataIdentifierValidatorObjReference with id 77 not found.
May 8, 2021 11:46:55 PM com.symantec.dlp.services.policy.task.PolicyPublisherTask addItemsToShippingPackage
WARNING: Content for object type DataIdentifierValidatorObjReference with id 14 not found.
May 8, 2021 11:46:55 PM com.symantec.dlp.services.policy.task.PolicyPublisherTask addItemsToShippingPackage
WARNING: Content for object type DataIdentifierValidatorObjReference with id 38 not found.
May 8, 2021 11:46:55 PM com.symantec.dlp.services.policy.task.PolicyPublisherTask addItemsToShippingPackage
WARNING: Content for object type DataIdentifierValidatorObjReference with id 27 not found.

When the above happens, in some cases it causes detection on those rules to fail.

Environment

In releases prior to 15.8, this was particularly noticeable in Cloud Detection Service, where the Validators will not load on a Cloud Detector (as confirmed by Technical Support).

Cause

There is a known defect which may relate to this issue, which occurred in previous releases of DLP:

"Collisions between the primary keys in the DI validator tables causes DI Validators to not be published"

This has been seen to happen when customers did have Custom DIs created, and there was some corruption in some of the Validators which crossed over to other DI rules in the policy tables.

But this also appears to happen if you policy import a policy into your Enforce Server from a higher version of DLP (which is neither a recommended nor supported practice).

Resolution

If possible, upgrade to DLP 15.8.

In that release, System DI Validators and Custom DI Validators now use the same sequence generator when adding new system validators to make sure the validators remain unique even when custom script validators exist.

If you are on a prior version, a fix for this issue must be obtained by opening a case with Technical Support.