When selecting a masking function to use, for a DB2 Table/column, the MOD11 function does not exist, even though it is listed on this page in the techdocs: Masking Functions and Parameters.
TDM Portal 4.11
Fast Data Masker 4.11
The MOD11 function is only available for the Mainframe platform, and not for Windows.
It seems that somehow the MOD11 function was added to the Reference - Masking Functions and Parameters section of the documentation by mistake, when it should only be listed in the Masking Functions for Mainframe section of the documentation.
If the DB2 database running on, z/OS (Mainframe), and you have the Mainframe add-on masking processes installed on your Mainframe, you could use the masking on the Mainframe platform to do this.