TDM Portal: PII Detection and Masking in Unstructured Text Data

Products

• CA Test Data Manager (Data Finder / Grid Tools)

Issue/Introduction

Customers frequently request the ability to automatically identify and mask Personally Identifiable Information (PII)—such as names, financial data, and organizational entities—within unstructured or plain text files (e.g., call summaries, chat logs, or free-text notes).

Environment

• Product: CA Test Data Manager (TDM)
• Components: TDM Portal, Fast Data Masker (FDM)
• Data Types: Unstructured plain text

Cause

CA Test Data Manager's PII scanning and masking logic is designed for structured or semi-structured data. The tool relies on identifying specific "fields" (columns, tags, or fixed-length segments) within a data source to apply masking functions. In unstructured text, PII does not reside in a predictable field, making it impossible for the current engine to natively differentiate between sensitive entities and general content without a delimiter or defined schema.

Resolution

Currently, automated PII discovery and redaction for unstructured/free-text is not a native feature of CA TDM, and would require a complete rewrite of the product.

Additional Information

Workarounds & Technical Approaches

1. Pattern-Based Recognition (RegEx): If the unstructured text contains predictable patterns (e.g., "Account Number: 12345"), you can create custom Data Classifiers using Regular Expressions (RegEx) to target those specific strings. However, this is less effective for variable entities like names or addresses. For more information about Custom Classifiers, please see Manage Data Classifiers.

2. Conversion to Structured Format: Preprocessing the unstructured data into a structured format (CSV or JSON) before ingestion into TDM allows the PII scan to function normally on the resulting fields.