Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
search cancel

Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient

book

Article ID: 295053

calendar_today

Updated On:

Products

Services Suite

Issue/Introduction

Symptoms:

A Hive job, probably launched from Oozie, fails with error

Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient

The complete stack trace is below:

Error Message

java.lang.RuntimeException: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
        at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:522)
        at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:677)
        at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:621)
        at org.apache.oozie.action.hadoop.HiveMain.runHive(HiveMain.java:306)
        at org.apache.oozie.action.hadoop.HiveMain.run(HiveMain.java:290)
        at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:47)
        at org.apache.oozie.action.hadoop.HiveMain.main(HiveMain.java:68)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:236)
        at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
        at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:453)
        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
        at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:422)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
        at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
Caused by: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
        at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1533)
        at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.(RetryingMetaStoreClient.java:86)
        at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:132)
        at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104)
        at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3043)
        at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3062)
        at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:503)
        ... 19 more
Caused by: java.lang.reflect.InvocationTargetException
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
        at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1531)
        ... 25 more
Caused by: java.lang.NoSuchMethodError: org.apache.thrift.protocol.TProtocol.getScheme()Ljava/lang/Class;
        at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$set_ugi_args.write(ThriftHiveMetastore.java)
        at org.apache.thrift.TServiceClient.sendBase(TServiceClient.java:63)
        at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.send_set_ugi(ThriftHiveMetastore.java:3682)
        at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.set_ugi(ThriftHiveMetastore.java:3673)
        at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.open(HiveMetaStoreClient.java:437)
        at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.(HiveMetaStoreClient.java:236)
        at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.(SessionHiveMetaStoreClient.java:74)
        ... 30 more

 

This article is only applicable if the following error is observed as a part of the stack trace:

java.lang.NoSuchMethodError: org.apache.thrift.protocol.TProtocol.getScheme
 

Environment


Cause

There is a JAR in the Java classpath that includes the class TProtocol but doesn't have the method "getScheme." This JAR is probably from a custom application.

Resolution

Option A: If your team knows what JAR is loading this class, change the "scope" in the "pom.xml" file to "provided" and re-deploy the JAR

Option B: If your team doesn't know what JAR is causing this issue, add the following property to your Job

<property>
   <name>oozie.launcher.mapreduce.map.java.opts</name>
   <value>-verbose</value>
</property>
  1. Re-run the job and reproduce the error
  2. Fetch the logs into a file
  3. Grep the logs for "TProtocol"
  4. Search for an output similar to this
container_e16_1483434141462_118653_01_000002.stdout:[Loaded org.apache.thrift.protocol.TProtocol from file:/grid/hadoop/yarn/local/filecache/6013/forBlogPosts-1.0-SNAPSHOT.jar] 
container_e16_1483434141462_118653_01_000002.stdout:[Loaded org.apache.thrift.protocol.TProtocolException from file:/grid/hadoop/yarn/local/filecache/6013/forBlogPosts-1.0-SNAPSHOT.jar] 
container_e16_1483434141462_118653_01_000002.stdout:Caused by: java.lang.NoSuchMethodError: org.apache.thrift.protocol.TProtocol.getScheme()Ljava/lang/Class;

Now that you know what JAR files are causing trouble, apply the change in the pom.xml mentioned in Option A, or remove this JAR from your classpath.


Powered by Wolken Software