Masking Option PROCESSCOUNT is not honored when LARGETABLESPLIT is enabled

Products

• CA Test Data Manager (Data Finder / Grid Tools)

Issue/Introduction

While doing testing of Masking Functions in TDM Portal 4.10 we noticed that when the LARGETABLE SPLIT is enabled, the PROCESSCOUNT option is not honored. IF we set the PROCESSCOUNT=100 we expect to only see 100 rows masked, however in our testing we are getting a random number of rows masked.

For example: 

BATCHSIZE=50000
COMMIT=50000
DIAGLEVEL=0
DROPRESTART=N
EMPTYASNULL=N
FETCHSIZE=150000
FORMATENCRYPT1DELIMITER=SPACE
FORMATENCRYPTEXETENDEDCHARS=Y
HEAPSIZE=50000
HIDERESTARTCOLUMN=N
INDEXRESTARTCOLUMN=Y
LARGETABLESPLITENABLED=Y
LARGETABLESPLITSIZE=3000000
MAKEPARTITIONS=Y
MD5HASHLOV=Y
OLDEXTENDEDCHARS=Y
ORACLEUSEROWNID=Y
ORDERBY=Y
OUTPUTIGNOREDLINES=Y
PARALLEL=8
PROCESSCOUNT=100
RESETRESTART=N
RESTART=Y

We see that 1000 of 26376296 rows were masked when we had set the Process Count to 100 rows. However, when we check the Oracle table, using the following SQL statement we see only 800 lines were actually masked.

select de_ident_ind, count(*) from <TABLE> group by de_ident_ind;

Why did Portal say 800 were masked?

Environment

Release : 4.10

Component :

Resolution

When PARALLEL is enabled, the expected behavior is to mask the number of rows set in PROCESSCOUNT option times the number of threads set by the PARALLEL option (PROCESSCOUNT x PARALLEL).