在Hadoop中包含第三方Jars [英] Include Third Party Jars in Hadoop

问题描述

我是Hadoop的新手。我已将Gson API添加到我的MapReducing程序中。当我运行程序时;

I am new to Hadoop. I have added Gson API to my MapReducing Program. When I am running the program getting;

Error: java.lang.ClassNotFoundException: com.google.gson.Gson

任何人都可以向我推荐如何将第三方库添加到Hadoop中？

Can anybody suggest me to how to add Third Party Libraries to Hadoop?

推荐答案

一定要添加任何相关性，以两个 HADOOP_CLASSPATH 和 -libjars 在提交一份工作时，如下面的例子所示：

Be sure to add any dependencies to both the HADOOP_CLASSPATH and -libjars upon submitting a job like in the following examples:

使用以下命令添加所有来自当前和 lib 目录：

Use the following to add all the jar dependencies from current and lib directories:

export HADOOP_CLASSPATH=$HADOOP_CLASSPATH:`echo *.jar`:`echo lib/*.jar | sed 's/ /:/g'`

请注意，当通过 hadoop jar 您还需要通过使用 -libjars 将它传递给任何依赖关系的罐子。我喜欢使用：

Bear in mind that when starting a job through hadoop jar you'll need to also pass it the jars of any dependencies through use of -libjars. I like to use:

hadoop jar <jar> <class> -libjars `echo ./lib/*.jar | sed 's/ /,/g'` [args...]

sed 命令需要不同的分隔符; HADOOP_CLASSPATH 为：分隔， -libjars 需要将，分开。

NOTE: The sed commands require a different delimiter character; the HADOOP_CLASSPATH is : separated and the -libjars need to be , separated.

这篇关于在Hadoop中包含第三方Jars的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！