如何在Hive中包含jar(Amazon Hadoop env) [英] How to include jars in Hive (Amazon Hadoop env)

问题描述

我需要在Hive中包含较新的protobuf jar(高于2.5.0).无论如何，无论我把罐子放在哪里-它都被推到类路径的末尾.我如何确保该jar位于Hive的类路径的开头?

I need to include newer protobuf jar (newer than 2.5.0) in Hive. Somehow no matter where I put the jar - it's being pushed to the end of the classpath. How can I make sure that the jar is in the beginning of the classpath of Hive?

推荐答案

要将您自己的jar添加到Hive类路径中，使其包含在类路径的开头，并且不会被某些hadoop jar重载，则需要设置以下Env变量-

To add your own jar to the Hive classpath so that it's included in the beginning of the classpath and not overloaded by some hadoop jar you need to set the following Env variable -

导出HADOOP_USER_CLASSPATH_FIRST = true

export HADOOP_USER_CLASSPATH_FIRST=true

这表明HADOOP_CLASSPATH将比普通的hadoop jars具有更高的优先级.

This indicates that the HADOOP_CLASSPATH will gain priority over general hadoop jars.

在Amazon emr实例上，您可以将其添加到/home/hadoop/conf/hadoop-env.sh，并在此文件中修改类路径.

At Amazon emr instances you can add this to /home/hadoop/conf/hadoop-env.sh, and modify the classpath in this file also.

当您想重载hadoop通用类路径随附的protobuf之类的jar时，这很有用.

This is useful when you want to overload jars like protobuf that come with the hadoop general classpath.

这篇关于如何在Hive中包含jar(Amazon Hadoop env)的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！