通过Cygwin在Windows上进行Hadoop配置 [英] Hadoop Configuration on Windows through Cygwin

问题描述

我试图在我的Windows 7机器上配置Hadoop。我可以开始命名节点和其他服务，但因为我正在运行的Hadoop包（版本1.0.3）附带的示例，以下错误来了：

I am trying to configure Hadoop on my Windows 7 machine. I am able to start name node and other services, but as I am running an example which comes with the Hadoop package (version 1.0.3), following error is coming:

bin/hadoop: line 320 : C:\Program: Command not found. 

我使用以下命令运行示例：

I ran the example with the following command:

bin/hadoop jar hadoop-examples-1.0.3.jar pi 10

我打开这个 hadoop 文件，其中错误来了，发现在第320行中
a正在生成路径：

I opened this hadoop file in which error is coming and found that in line 320 a path is being generated:

JAVA_PLATFORM=`CLASSPATH=${CLASSPATH} ${JAVA} -Xmx32m ${HADOOP_JAVA_PLATFORM_OPTS} org.apache.hadoop.util.PlatformName | sed -e "s/ /_/g"`

所以我觉得问题可能在于这个 JAVA 变量，因为Cygwin对路径名使用不同的约定。有没有人也遇到过这个问题，或者知道是什么原因造成的？

So I am feeling that problem might be in this JAVA variable as Cygwin uses different conventions for path name. Has anyone also faced this problem or know what is causing the problem?

推荐答案

快速总结：

• （路径）/ bin / hadoop下的 hadoop bash脚本实际上有一个bug。该脚本假定hado需要的所有文件/路径都不具有空格。



Quick summary:

• The hadoop bash script under (path)/bin/hadoop actually has a bug in it. The script assumes that none of the files / paths that hadoop needs will have spaces in them. Well, for anything Windows, they will all have a space somewhere, since "Program Files" has a space in it.

这是一个棘手的问题...我碰到了同样的问题，花了我一会儿来解决。

This is a tricky one... I ran into the same problem and it took me a while to fix.

首先，问题：通过脚本设置环境变量可能会在文件路径/名称中涉及空格时出现粗略（这在当前的非* nix系统中经常出现）。

First, the problem: setting environment variables via scripts can get sketchy when spaces are involved in the file paths / names (which occurs fairly often in non-*nix systems these days).

接下来，您可能需要 两个 来解决问题：

Next, there are likely two places where you need to fix the problem:




1. 在（路径）/conf/hadoop-env.sh 脚本中，应该设置 JAVA_HOME 脚本，它应该看起来像：

1. In your (path)/conf/hadoop-env.sh script, you should be setting the JAVA_HOME script, and it SHOULD look something like:

export JAVA_HOME=/cygdrive/c/"Program Files"/Java/jdk1.7.0_06

（注意Program Files ，因此它被识别为单个元素。不能使用 \ 转义字符，因为cygwin对Windows的UNIX路径执行了一些操作，因此

(Note that there are quotation marks around the "Program Files", so that it is recognized as a single element. You cannot use the \ escape character because cygwin does some finagling of Windows to UNIX paths, so the \ cannot act as escape.

在（路径）/ bin / hadoop 脚本，第320行很可能写成如下：

In your (path)/bin/hadoop script, line 320 is likely written something like the following:

JAVA_PLATFORM=`CLASSPATH=${CLASSPATH} ${JAVA} -Xmx32m ${HADOOP_JAVA_PLATFORM_OPTS} org.apache.hadoop.util.PlatformName | sed -e "s/ /_/g"`

您需要将其更改为：

JAVA_PLATFORM=`CLASSPATH="${CLASSPATH}" "${JAVA}" -Xmx32m ${HADOOP_JAVA_PLATFORM_OPTS} org.apache.hadoop.util.PlatformName | sed -e "s/ /_/g"`

请注意，环境变量 $ {CLASSPATH} 和 $ {JAVA} 。通过在其周围加上引号，您表示该变量指定的整个字符集应被视为一个字符串对象。

Note that I have added quotation marks around the environment variables ${CLASSPATH} and ${JAVA}. By putting the quotation marks around it, you are saying that "the entire set of characters specified by this variable should be considered one string object".





---





好的，现在如果你想知道为什么会发生这种情况和发生了什么，问题是你的JDK可能存储在Program Files或者在程序文件（x86）下，它们在路径中都有 空格 。 Hadoop所需的所有其他环境变量不依赖于Program Files路径中的任何内容。所以这就是为什么你只看到一个错误被标记。所有其他缺少引号的环境变量都没有空格。

---

OK, now if you care to understand why this is happening and what's going on, the problem is that your JDK is likely stored under "Program Files", or maybe under "Program Files (x86)", both of which have spaces within the path. All the other environment variables that Hadoop needs are not dependent upon anything within the "Program Files" pathway. So that's why you only see the one error being flagged. All the other environment variables which are missing the quotes simply don't have spaces within them.

这篇关于通过Cygwin在Windows上进行Hadoop配置的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！