Pre/Post valid characters for Custom Data Identifiers are overwritten after upgrading 16.0 to 16.0 RU
Article ID: 275654
Updated On:
Products
Issue/Introduction
Pre/Post valid characters for Custom Data Identifiers (DI) are overwritten after upgrading to DLP 16.0 RU
Important: This only occurs for Custom Data Identifiers pre and post validators that were created and/or edited on DLP software version 16.0 and then upgraded to 16.0 RU (16.0.10000.60631).
Environment
Upgrade paths:
15.8 -> 16.0 -> 16.0.1 (16.0 RU)
16.0 -> 16.0.1 (16.0 RU)
Cause
Background:
DLP 16.0 simplified the creation of Custom Data Identifiers by obsoleting the need to specify pre and post invalid characters.
Custom Data Identifiers in DLP 16.0 rely upon pre/post valid characters for pattern validation and eliminates the need to specify pre/post invalid characters. As such, the upgrade to DLP 16.0 converts the 4 part design, pre/post valid/invalid characters, into the new 2 part design, pre/post valid characters.
When a new Custom Data Identifier (in DLP 16.0) is created, or an existing one edited, the pre and post invalid character strings, no longer used for 16.0, gets set to NULL.
The issue occurs when migrating from 16.0 to 16.0 RU. The RU migrator erroneously re-runs the pre/post character migration, converting the NULL in the invalid character strings to return a string of characters for the valid character string (shown below), thus overwriting the pre/post valid characters.
!"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\\]^_`abcdefghijkmopqrstvwxyz{|}~
Impact:
Upgrading to 16.0 RU, when going through DLP 16.0, overwrites Pre/Post validation characters for Custom Data Identifiers which may cause undesirable detection results.
Resolution
For customers on 16.0 and not already upgraded to RU
Broadcom has released an updated 16.0 RU installer that addresses the issue. Download the updated platform installation binaries from the Broadcom Support Portal and perform the upgrade.
If you have installed 16.0.1 (RU1) using binaries timestamped in September (original GA), but you have not completed the upgrade by running the Migration, uninstall Symantec Data Loss Prevention version 16.0.1, then use the latest version binaries on the Support Portal:
- Old installer has build # 16.0.10000.60631, which was made available on 8 September 2023.
- New installer has build # 16.0.10000.60678, which was made available for download starting on 5 November 2023.
New build MD5 checksums:
- Windows: 08647c9378312f2d99766c5ff5bef628
- Linux: 43a58046b5a20933851b1347dff5ba47
For customers already on 16.0 RU
The following SQL shows the custom DIs that were overwritten and the policies.
SELECT di.name,di.dataidentifierid,b.breadthid,count(distinct pc.policyid) PolicyCnt
FROM dataidentifier di
JOIN dataidentifierbreadth b ON b.dataidentifierid=di.dataidentifierid
LEFT OUTER JOIN dataidentifiercondition dc on dc.breadthid=b.breadthid
LEFT OUTER JOIN leafconditionsinpolicyview pc
on pc.conditionid=dc.conditionid and pc.policyisdeleted=0
WHERE di.isdeleted=0 AND di.identifiertype='CUSTOM'
AND ((postinvalidcharacters IS NULL
AND
length(postvalidcharacters)>90)
OR (preinvalidcharacters IS NULL
AND
length(prevalidcharacters)>90))
GROUP BY di.name,di.dataidentifierid,b.breadthid;
SELECT di.name CustomDI,pc.policyname,c.name
Condtion,pc.conditiongrouptype,DECODE(pc.isorcondition,0,1,0) isException
FROM dataidentifier di
JOIN dataidentifierbreadth b ON b.dataidentifierid=di.dataidentifierid
JOIN dataidentifiercondition dc on dc.breadthid=b.breadthid
JOIN leafconditionsinpolicyview pc on pc.conditionid=dc.conditionid and
pc.policyisdeleted=0
JOIN conditionview c on c.conditionid=pc.conditionid
WHERE di.isdeleted=0 AND di.identifiertype='CUSTOM'
AND ((postinvalidcharacters IS NULL
AND
length(postvalidcharacters)>90)
OR (preinvalidcharacters IS NULL
AND
length(prevalidcharacters)>90))
ORDER BY di.name,pc.policyname,c.name;
If this Enforce was upgraded from 15.8 to 16.0, before upgrading to 16.0 RU, the following path contains the original correct update of the Custom Data Identifiers that can be ran to update the pre/post valid chars to their correct strings.
\DataLossPrevention\EnforceServer\16.0.00000\Protect\Migrator\SQL\update_di_prepost_valid_characters.sql
The following SQL will reset any remaining erroneous valid pre/post character string to the default pre/post valid character string which will require a restart of the Monitor Controller after SQL is used to update these custom DIs.
UPDATE dataidentifierbreadth
SET prevalidcharacters = ',=:#"''()>;@!`~$%^*'
WHERE preinvalidcharacters IS NULL
AND
length(prevalidcharacters)>90
AND dataidentifierid IN
(SELECT dataidentifierid FROM dataidentifier WHERE identifiertype='CUSTOM');
UPDATE dataidentifierbreadth
SET postvalidcharacters = ',."''()<;&=@`~'
WHERE postinvalidcharacters IS NULL
AND
length(postvalidcharacters)>90
AND dataidentifierid IN
(SELECT dataidentifierid FROM dataidentifier WHERE identifiertype='CUSTOM');
UPDATE Dataidentifier
SET VERSIONUUID=LOWER(regexp_replace(rawtohex(sys_guid()),
'([A-F0-9]{8})([A-F0-9]{4})([A-F0-9]{4})([A-F0-9]{4})([A-F0-9]{12})','\1-\2-\3-\4-\5'))
WHERE identifiertype='CUSTOM';
Customers can additionally edit the Custom Data Identifiers to restore the original list of pre/post valid characters (referenced from a DLP 16.0 backup) instead of the default values.
Feedback
