Need an approach to mask a flat file with variable record details

Products

• CA Test Data Manager (Data Finder / Grid Tools)

Issue/Introduction

We have requirements for masking several files that are all fixed format files with:

• 1 Header row and
• 1 Trailer Row
•  15 record Types that contain 1 or more fields that need to be masked
• 4 Record Types that contain no field/s to be masked

contain several record types with possibility of having different  PII/PHI elements in a different location depending upon the Record Type field which is the 6th position (3 chr). 

Kindly provide guidance on how to go about such. In the folder we have over 5000 such files to be masked and processed after masking. 

So, as an extension to our solution, kindly suggest a way to "automate" the process. So, when a file is dropped in a folder, it would trigger the FDM, with reference to a common _DM file and after processing is completed - the original and scramble are moved to another location. 

Environment

Release : 4.10

Resolution

Here is an example of masking a file with multiple record types.

So in the file, Emp_data.dat, there are 3 record types.

employee

employee address

employee creditcard

 

You need to create a .dm.txt file for each record type.  In each .dm.txt file you define how FDM should determine which record is which?

for example  in the employee.dm.txt file you see

 

DELIM=FIXED,RECTYPE=RECORD_ID:"EMP"
RECORD_ID,3
EMPLOYEE_ID,4
FIRST_NAME,20
LAST_NAME,20
EMAIL,20
PHONE_NUMBER,15
BIRTH_DATE,10,"YYYY-MM-DD"
EMP_SSN,10
EMP_PASSWORD,20
HIRE_DATE,10,"YYYY-MM-DD"
SALARY,5
EXPENSES,5
SAVINGS,5
EMP_RECORD_ID,8

 

What is in bold drives this.

 

It doesn't have to be the first column.

 

Once this is all done, you have to create or update the connection file .txt file to include the multiple .dm.txt files.  When you connect, FDM will show each .dm.txt file as if it is a table.

 

Note: you do not need to include the Header, and Trailer. All that is required is the DELIM and RECTYPE. Also, FDM doesn't handle spaces well, so if you have a space (for example, First Name) you will want to delete the space or replace it with an underscore. 

 

For automating the process, once the FDM configuration is in place, you can design a Javelin Workflow to check the file location where the .dm.txt files will be placed, and kickoff a masking job, if any files have been added to the directory. Once masking completes, as a post process, Javelin can move/archive the processed files to another directory.