通过 Cygwin 在 Windows 上配置 Hadoop [英] Hadoop Configuration on Windows through Cygwin

问题描述

我正在尝试在我的 Windows 7 机器上配置 Hadoop.我能够启动名称节点和其他服务，但是当我运行 Hadoop 包(版本 1.0.3)附带的示例时，出现以下错误:

I am trying to configure Hadoop on my Windows 7 machine. I am able to start name node and other services, but as I am running an example which comes with the Hadoop package (version 1.0.3), following error is coming:

bin/hadoop: line 320 : C:Program: Command not found. 

我使用以下命令运行示例:

I ran the example with the following command:

bin/hadoop jar hadoop-examples-1.0.3.jar pi 10

我打开了这个 hadoop 文件，其中出现了错误，发现在第 320 行正在生成路径:

I opened this hadoop file in which error is coming and found that in line 320 a path is being generated:

JAVA_PLATFORM=`CLASSPATH=${CLASSPATH} ${JAVA} -Xmx32m ${HADOOP_JAVA_PLATFORM_OPTS} org.apache.hadoop.util.PlatformName | sed -e "s/ /_/g"`

所以我觉得问题可能出在这个 JAVA 变量中，因为 Cygwin 对路径名使用不同的约定.有没有人也遇到过这个问题或者知道是什么原因造成的?

So I am feeling that problem might be in this JAVA variable as Cygwin uses different conventions for path name. Has anyone also faced this problem or know what is causing the problem?

推荐答案

快速总结:

• (path)/bin/hadoop 下的hadoop bash 脚本实际上有一个bug.该脚本假定 hadoop 需要的所有文件/路径中都没有空格.好吧，对于任何 Windows，它们都会在某个地方有一个空格，因为程序文件"中有一个空格.

Quick summary:

• The hadoop bash script under (path)/bin/hadoop actually has a bug in it. The script assumes that none of the files / paths that hadoop needs will have spaces in them. Well, for anything Windows, they will all have a space somewhere, since "Program Files" has a space in it.

这是一个棘手的问题...我遇到了同样的问题，我花了一段时间才解决.

This is a tricky one... I ran into the same problem and it took me a while to fix.

首先，问题是:当文件路径/名称中包含空格时，通过脚本设置环境变量可能会变得粗略(这在当今的非 *nix 系统中经常发生).

First, the problem: setting environment variables via scripts can get sketchy when spaces are involved in the file paths / names (which occurs fairly often in non-*nix systems these days).

接下来，您可能需要在 两个处解决问题:

Next, there are likely two places where you need to fix the problem:

1. 在您的 (path)/conf/hadoop-env.sh 脚本中，您应该设置 JAVA_HOME 脚本，它应该类似于:

1. In your (path)/conf/hadoop-env.sh script, you should be setting the JAVA_HOME script, and it SHOULD look something like:

export JAVA_HOME=/cygdrive/c/"Program Files"/Java/jdk1.7.0_06

(请注意，Program Files"周围有引号，以便将其识别为单个元素.您不能使用 转义字符，因为 cygwin 对 Windows 进行了一些对 UNIX 的修改路径，所以 不能作为转义符.

(Note that there are quotation marks around the "Program Files", so that it is recognized as a single element. You cannot use the escape character because cygwin does some finagling of Windows to UNIX paths, so the cannot act as escape.

在您的 (path)/bin/hadoop 脚本中，第 320 行可能会写如下内容:

In your (path)/bin/hadoop script, line 320 is likely written something like the following:

JAVA_PLATFORM=`CLASSPATH=${CLASSPATH} ${JAVA} -Xmx32m ${HADOOP_JAVA_PLATFORM_OPTS} org.apache.hadoop.util.PlatformName | sed -e "s/ /_/g"`

您需要将其更改为:

JAVA_PLATFORM=`CLASSPATH="${CLASSPATH}" "${JAVA}" -Xmx32m ${HADOOP_JAVA_PLATFORM_OPTS} org.apache.hadoop.util.PlatformName | sed -e "s/ /_/g"`

请注意，我在环境变量 ${CLASSPATH} 和 ${JAVA} 周围添加了引号.通过将引号括起来，您的意思是应将此变量指定的整个字符集视为一个字符串对象".

Note that I have added quotation marks around the environment variables ${CLASSPATH} and ${JAVA}. By putting the quotation marks around it, you are saying that "the entire set of characters specified by this variable should be considered one string object".

<小时>

好的，现在如果您想了解为什么会发生这种情况以及发生了什么，问题是您的 JDK 可能存储在程序文件"下，或者可能存储在程序文件 (x86)"下，两者都有空格.Hadoop 需要的所有其他环境变量不依赖于程序文件"路径中的任何内容.这就是为什么您只看到一个错误被标记的原因.所有其他缺少引号的环境变量都没有空格.

---

OK, now if you care to understand why this is happening and what's going on, the problem is that your JDK is likely stored under "Program Files", or maybe under "Program Files (x86)", both of which have spaces within the path. All the other environment variables that Hadoop needs are not dependent upon anything within the "Program Files" pathway. So that's why you only see the one error being flagged. All the other environment variables which are missing the quotes simply don't have spaces within them.

这篇关于通过 Cygwin 在 Windows 上配置 Hadoop的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！