Numerical values with punctuation create EDM false positives

Products

• Data Loss Prevention

Issue/Introduction

You are seeing false positive matches in EDM incidents where the value in the incident matches an entry from the EDM data source column in value, but not in syntax. 

For example, the value in the indexed data source is:

123456

Whereas the false positive matches are generated for the following variations:

123,456

123.456

This can be also seen with longer digits and only if the punctuation is placed at specific positions in the digit as present in the processed message - as in the below example patterns:

AAA.BBB

AAA.BBB.CCC

AAA.BBB.CCC.DDD

Basically the separator comes every three digits from the end, following a common format of financial values where every block of three digits is separated from the other with a dot or a comma. 

Cause

This is caused by a built-in validator in the EDM processing engine which normalizes the values and removes the punctuation before passing the value to detection processing. Taking the above example, 456.789 becomes 456789 and as a result will exactly match the value from the data source. 

Resolution

It's not possible to disable the normalizer, as confirmed with Engineering. Tests were done to check whether it's possible to circumvent it by combining multiple detection rules with an AND, i.e. :

EDM + custom Data Identifier (designed to only detect six-digit strings with a RegEx \d{6})

This however produces more matches and does not eliminate the FP detections of EDM. 

For example, below test strings:

123456
789012
456.789
012.345

Will produce a total of 6 matches:

-4 EDM matches (2 for 6-digit strings without punctuation - 123456 and 789012 - and 2 for 6-digit strings with punctuation that looks like 'financial' syntax - 456.789 and 012.345)
-2 DI matches (only for the 6-digit strings without punctuation - 123456 and 789012)

 

Additional Information

See also: EDM results are not matching as expected for source data - More details about how that EDM validator works when parsing values with delimiters.