After upgrading to DLP 16.0 creation of EMDI index fails

Products

• Data Loss Prevention
• Data Loss Prevention Enforce

Issue/Introduction

After upgrading to DLP 16.0 creating of EMDI index may fail with the following exception: Error: Data file was corrupted (Creating Index Version X):

At the same time the following error can be observed in tomcat (localhost) log, replace <Name of the EMDI Index> with the name of the index which in this situation would be "Test EMDI".

SEVERE [com.vontu.profiles.manager.database.DataSourceIndexCreator] Indexing error occurred while creating exact data profile "<Name of the EMDI index>". Too many invalid patterns

Environment

Release : 16.0

Cause

This is a known issue with DLP 16.0 where EMDI indexing will fail if the Data Identifier when the Legacy pattern of the DI is empty.

Resolution

The issue will be addressed in DLP 16.0 RU (Release Update). In the meantime the resolution to the issue is to populate the Legacy field with a valid regex pattern that the Data Identifier should match. The same pattern that worked in the previous version, for example 15.8, can be used.

This has an additional benefit as starting in DLP 16.0 a new regex engine was released that the older agents are not compatible with. DLP 16.0 agents will use the new pattern (on the left), while older agents which do not use the new engine will use the pattern present in the Legacy field. This is especially important during the upgrade transition period where the same environment may still be running older agents along side upgraded agents.

 

Please note that the new pattern field and the Legacy field has different specifications and so the patterns should be built around those. For more details refer to the documentation linked below.

Legacy pattern specification:

Legacy data identifier pattern language specification (broadcom.com)

New pattern specification:

Limitations of data identifier support for PCRE regular expressions (broadcom.com)