FDM is changing non-Nullable columns that contain a space value to a NULL value

book

Article ID: 140661

calendar_today

Updated On:

Products

CA Test Data Manager (Data Finder / Grid Tools)

Issue/Introduction

 We have an Oracle Database where some columns were configured to be non-Nullable, and data with no values are inserted into those columns as a String containing a space as the value. However, FDM is not handling these columns as expected. We are seeing the data in these columns as NULL values and not the intended space value.

 

Cause

Some columns were configured to be non-Nullable and data with no values were inserted into those columns as a string with a space. This is an inefficient way to store records and it would have made sense to set the column as Nullable and any data with no value to be stored as NULL. This is the reason why other customers do not experience this same problem.


Engineering assumes there is a business case to have such a database design. Our FDM application trims (removes spaces) the data before it is masked. This means that records that are defined as 'string with a space' will become empty.


When we reinsert the processed data back into the database, we would be inserting an empty string.


Oracle database (not like other databases) does not have a concept of an empty string and therefore it will convert it to NULL string. Inserting a NULL string into a non-Nullable column would result in the issue reported.

Environment

Release : 4.8

Component : CA Test Data Manager - Fast Data Masker

Resolution

 

 

Our Engineering team has developed an option in the FDM application to stop it from trimming the data before processing. The new option is TRIMVALUES, which can be disabled (N). The default TRIMVALUES is Yes (Y).




This is a risky endeavor as it means that some masking functions could be adversely affected. To help explain the new engine inner workings, Engineering has also included the high level diagram about:


Masking engines containing FDM processes
FDM processes using multiple Thread processing blocks when large table splitting is enabled



This new FDM option is available in FastDataMasker-4.8.145.0.


 

Attachments