我如何编译/使用hadoop 2.0的mahout？ [英] how can I compile/using mahout for hadoop 2.0?

问题描述

最新发布的mahout 0.9只建立在hadoop 1.x上。 （mvn clean install）
如何编译​​hadoop 2.0.x的mahout？

因为当我运行命令时：

hadoop jar mahout-examples -0.9-快照job.jar org.apache.mahout.cf.taste.hadoop.item.RecommenderJob -s SIMILARITY_COOCCURENCE -i测试-o结果

我总是得到错误消息

IncompatibleClassChangeError：实测值接口org.apache.hadoop.mapreduce.JobContext，但类是预计。

谢谢！

解决方案

编译Mahout以使用2.x，因为它没有在与Hadoop 2.x兼容的包中发布：

  mvn clean install -Dhadoop2 -Dhadoop2.version = 2.2.0 -DskipTests = true 
  

如果你想在构建之后确认引入了正确的依赖关系，从项目的根目录运行以下内容：

  find 。 -name hadoop * .jar 
  

上述命令生成的工件与0.9版本中的工件不同，所以你需要更新的不仅仅是版本号（似乎有一个新的mahout-mrlegacy jar）。这就是说，如果有人能指出我使用2.x运行的官方产品（即使它是SNAPSHOT），我会很感激的。

p>

The latest release mahout 0.9 is only built on hadoop 1.x. (mvn clean install) How can I compile mahout for hadoop 2.0.x?

Because When I was running the commands:

hadoop jar mahout-examples-0.9-SNAPSHOT-job.jar org.apache.mahout.cf.taste.hadoop.item.RecommenderJob -s SIMILARITY_COOCCURENCE -i test -o result

I always got the error message

IncompatibleClassChangeError: Found interface org.apache.hadoop.mapreduce.JobContext, but class was expected.

Thanks!

解决方案

To compile Mahout to work with 2.x since it isn't released in a package that is compatible with Hadoop 2.x:

mvn clean install -Dhadoop2 -Dhadoop2.version=2.2.0 -DskipTests=true

If you want to confirm after the build that the correct dependencies were brought in, run the following from the root of the project:

find . -name hadoop*.jar

The artifacts generated by the above command differ from what's in the 0.9 release so you'll need to update more than just the version number (there seems to be a new mahout-mrlegacy jar).

That being said, if someone can point me to an official artifact that runs with 2.x (even if it's a SNAPSHOT), I'd appreciate it.

这篇关于我如何编译/使用hadoop 2.0的mahout？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！