通过分布式缓存hadoop添加jar [英] Adding jar via distributed cache hadoop

问题描述

使用Disributed Cache添加.jar文件到类路径中：

  DistributedCache.addFileToClassPath（new Path（二进制/ tools.jar），job.getConfiguration（））; 
  

我不确定addFileToClassPath（）是否用于添加.jar文件的正确API班级路径。当我尝试从映射器中检索类路径时，我看不到添加的jar。类路径包含作业的工作目录（jobcache目录），但不包括通过分布式缓存分发的jar。

 属性prop = System.getProperties（）; 
 System.out.println（classpath is：+ prop.getProperty（java.class.path，null））; 
  

我也尝试了addArchiveToClassPath（）..它没有工作..

我是否缺少什么？

谢谢，

解决方案问题在于路径。 addFileToClassPath（）或 addArchiveToClassPath（）仅以绝对路径作为输入。二进制/ tools.jar是相对的，因此没有工作。我需要指定路径为 /user/<username>/binary/tools.jar .. 现在它工作正常。即使 hdfs：//< hostname>：port / user / .. 失败。

谢谢大家。 。

I am adding a .jar file to the class path using Disributed Cache:

DistributedCache.addFileToClassPath(new Path("binary/tools.jar"), job.getConfiguration());

I am not sure whether addFileToClassPath() is the correct API to be used for adding .jar files to the class path. When I try to retrieve the class path from the mapper, I could not see the added jar. The classpath contains the working directory for the job (jobcache dir), but that does not include the jar distributed through Distributed Cache.

Properties prop = System.getProperties();
System.out.println("The classpath is: " + prop.getProperty("java.class.path", null));

I tried addArchiveToClassPath() too.. It did no work..

Am I missing something ?

Thanks,

解决方案

The problem was was with the path. addFileToClassPath() or addArchiveToClassPath() takes only absolute path as input. binary/tools.jar is relative and hence did not work. I need to specify the path as /user/<username>/binary/tools.jar.. Now it works fine. Even hdfs://<hostname>:port/user/.. fails.

Thank you all..

这篇关于通过分布式缓存hadoop添加jar的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！