通过分布式缓存hadoop添加jar [英] Adding jar via distributed cache hadoop
问题描述
DistributedCache.addFileToClassPath(new Path(二进制/ tools.jar),job.getConfiguration());
我不确定addFileToClassPath()是否用于添加.jar文件的正确API班级路径。当我尝试从映射器中检索类路径时,我看不到添加的jar。类路径包含作业的工作目录(jobcache目录),但不包括通过分布式缓存分发的jar。
属性prop = System.getProperties();
System.out.println(classpath is:+ prop.getProperty(java.class.path,null));
我也尝试了addArchiveToClassPath()..它没有工作..
我是否缺少什么?
谢谢,
addFileToClassPath()
或 addArchiveToClassPath()
仅以绝对路径作为输入。二进制/ tools.jar是相对的,因此没有工作。我需要指定路径为 /user/<username>/binary/tools.jar ..
现在它工作正常。即使 hdfs://< hostname>:port / user / ..
失败。 谢谢大家。 。
I am adding a .jar file to the class path using Disributed Cache:
DistributedCache.addFileToClassPath(new Path("binary/tools.jar"), job.getConfiguration());
I am not sure whether addFileToClassPath() is the correct API to be used for adding .jar files to the class path. When I try to retrieve the class path from the mapper, I could not see the added jar. The classpath contains the working directory for the job (jobcache dir), but that does not include the jar distributed through Distributed Cache.
Properties prop = System.getProperties();
System.out.println("The classpath is: " + prop.getProperty("java.class.path", null));
I tried addArchiveToClassPath() too.. It did no work..
Am I missing something ?
Thanks,
The problem was was with the path. addFileToClassPath()
or addArchiveToClassPath()
takes only absolute path as input. binary/tools.jar is relative and hence did not work. I need to specify the path as /user/<username>/binary/tools.jar..
Now it works fine. Even hdfs://<hostname>:port/user/..
fails.
Thank you all..
这篇关于通过分布式缓存hadoop添加jar的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!