Getting ClassNotFoundException: com.ctc.wstx.io.InputBootstrapper while executing Hive masking script

Products

• CA Test Data Manager (Data Finder / Grid Tools)

Issue/Introduction

We are getting the below error while executing the Hive masking script. Can you please check and update us as early as possible.

Java.lang.NoClassDefFoundError: com/ctc/wstx/io/InputBootstrapper
        at org.apache.hadoop.fs.FileSystem.<clinit>(FileSystem.java:206)
        at org.apache.hadoop.hive.conf.valcoersion.JavaIOTmpdirVariableCoercion.<clinit>(JavaIOTmpdirVariableCoercion.java:37)
        at org.apache.hadoop.hive.conf.SystemVariables.<clinit>(SystemVariables.java:37)
        at org.apache.hadoop.hive.conf.HiveConf$ConfVars.<init>(HiveConf.java:5830)
        at org.apache.hadoop.hive.conf.HiveConf$ConfVars.<init>(HiveConf.java:5811)
        at org.apache.hadoop.hive.conf.HiveConf$ConfVars.<clinit>(HiveConf.java:484)

Also we have noticed that when we trigger manually we are getting the below error but still its completing the execution.

SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/TDM/TDM_SYED/BIGDATA_COB/FDM_192/Fastdatamasker_simulate.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:TDM/TDM_SYED/BIGDATA_COB/FDM_192/lib/log4j-slf4j-impl-2.23.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/TDM/TDM_SYED/BIGDATA_COB/FDM_192/drivers/snowflake/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]

Environment

FDM 4.11.x

Cause

Missing Jars: 

With respect to the hive masking, in TDM, the database drivers are available under drivers folder in FDM HOME directory. For Hive they can be found under Hive, similarly for big query, snowflake, etc.

In default install location the hive jars are available @ C:\Program Files\Grid-Tools\FastDataMasker\drivers\hive

Some of the jars which are common may be available in the C:\Program Files\Grid-Tools\FastDataMasker\lib as well. The jar woodstox-core-*.jar is located under /lib directory.

If FDM is repackaged then we need to ensure that all necessary driver folders are packaged and the location is mentioned in the Classpath along with lib.

Sample Classpath can be seen below, which can be got from bat file we create for masking:

"C:\Program Files\Grid-Tools\FastDataMasker\jre/bin/java" -Dfile.encoding=UTF-8 -Djava.util.logging.config.file="C:/Program Files/Grid-Tools/FastDataMasker/logging.properties" -Xms100M -Xmx4096M  -XX:+HeapDumpOnOutOfMemoryError -Djava.library.path="%PATH%;C:/Program Files/Grid-Tools/FastDataMasker/SQLSERVER_DLLs/x64;C:\cspm\cloakware\cspmclient\lib;c:\cspm\cloakware\cspmclient\jre\bin;C:\Program Files\Grid-Tools\FastDataMasker\SQLSERVER_DLLs\x64" -Dcspm_client_config_file="C:\cspm\cloakware\cspmclient\config\cspm_client_config.xml" -cp ".;C:/Program Files/Grid-Tools/FastDataMasker/Fastdatamasker.jar;C:/Program Files/Grid-Tools/FastDataMasker/FDMCommon.jar;C:/Program Files/Grid-Tools/FastDataMasker/lib/*;C:/Program Files/Grid-Tools/FastDataMasker/drivers/cassandra/*;C:/Program Files/Grid-Tools/FastDataMasker/drivers/mongodb/*;C:/Program Files/Grid-Tools/FastDataMasker/drivers/bigquery/*;C:/Program Files/Grid-Tools/FastDataMasker/drivers/snowflake/*;C:/Program Files/Grid-Tools/FastDataMasker/drivers/hive/*;C:/Program Files/Grid-Tools/FastDataMasker/custom/*;" com.grid_tools.products.datamasker.Datamasker "C:/Users/AppData/Roaming/Grid-Tools/Fastdatamasker/connectHive.txt" "C:\Users\AppData\Roaming\Grid-Tools\Fastdatamasker\hivemasking.csv" "C:\Users\AppData\Roaming\Grid-Tools\Fastdatamasker\hivemasking_options.txt"

Resolution

Issue is resolved by adding the missing jars.