Hadoop分布式缓存类路径 [英] Hadoop DistributedCache classpath

问题描述

我正在考虑使用 DistributedCache 将作业代码分发到所有节点。我无法理解 addFileToClassPath（）和 addArchiveToClassPath（）之间的区别。从逻辑上讲，前者似乎是用于单个类文件，后者用于jar。但是在javadocs中，他们有这个示例代码：

  DistributedCache.addFileToClassPath（new Path（/ myapp / mylib.jar ），工作）; 
  

解决方案

question 可能会有所帮助

作为用户中提到的用户之一评论部分有一个与 addArchiveToClassPath（）相关的错误。学习和解决问题的最好方法是通过将Hadoop更新为1.0.0

从apache网站：
$ b

addArchiveToClassPath

code>将存档路径添加到当前的一组类路径条目。它也将档案添加到缓存中。归档文件将被解压缩，并在分发时添加到类路径中。
$ b addFileToClassPath 将文件路径添加到当前的一组类路径条目中它也将文件添加到缓存中。在添加到类路径中时，使用此方法添加的文件不会被解压缩。要将档案添加到classpath，请改用addArchiveToClassPath（Path）方法。

我意识到Hadoop文档是由不熟悉英文文法的人编写的。我明白你为什么感到沮丧。

I have a Hadoop 0.20.2 cluster.

I'm thinking of using DistributedCache to distribute the job code to all the nodes. I can't understand the difference between addFileToClassPath() and addArchiveToClassPath(). Logically, it would seem that the former is for single class files, and the latter is for jars. But right in the javadocs, they have this example code:

DistributedCache.addFileToClassPath(new Path("/myapp/mylib.jar"), job);

解决方案

This question could be helpful

as one of the users mentioned in the comment section there is a bug associated with addArchiveToClassPath() . the best way to learn and solve the problem is by updating your Hadoop to 1.0.0

From the apache website:

addArchiveToClassPath Add an archive path to the current set of classpath entries. It adds the archive to cache as well. Archive files will be unpacked and added to the classpath when being distributed.

addFileToClassPath Add an file path to the current set of classpath entries It adds the file to cache as well. Files added with this method will not be unpacked while being added to the classpath. To add archives to classpath, use the addArchiveToClassPath(Path) method instead.

I have realized that the Hadoop document is written by somebody who does not know english grammer very well. I see why you are frustrated.

这篇关于Hadoop分布式缓存类路径的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！