Spark 1.6-未能在hadoop二进制路径中找到winutils二进制文件 [英] Spark 1.6-Failed to locate the winutils binary in the hadoop binary path

问题描述

我知道有一个与这个非常相似的帖子(

看完这篇文章后无法在 hadoop 二进制文件中找到 winutils 二进制文件路径

我意识到我需要 winutils.exe 文件，所以我用它下载了一个 hadoop 二进制 2.6.0，定义了一个名为 HADOOP_HOME 的环境变量:

 值为 C:UsersGERALDesktophadoop-2.6.0in

然后像这样把它放在 Path 上:%HADOOP_HOME%

但是当我尝试代码时仍然出现相同的错误.有谁知道如何解决这个问题?

解决方案

如果您使用 Hadoop 在 Windows 上运行 Spark，那么您需要确保 Windows hadoop 安装正确安装.要运行 spark，您需要在您的 hadoop 主目录 bin 文件夹中有 winutils.exe 和 winutils.dll.

我会请你先试试这个:

1) 您可以从以下链接中的包中下载 .dll 和 .exe 文件.

https://codeload.github.com/sardetushar/hadooponwindows/zip/master

2) 将 winutils.exe 和 winutils.dll 从该文件夹复制到您的 $HADOOP_HOME/bin.

3) 在 spark-env.sh 或命令中设置 HADOOP_HOME，并将 HADOOP_HOME/bin 添加到 PATH.

然后尝试运行.

如果你需要hadoop安装帮助的任何帮助，有一个很好的链接，你可以试试.

http:///toodey.com/2015/08/10/hadoop-installation-on-windows-without-cygwin-in-10-mints/

但是，那可以等.您可以尝试前几个步骤.

I know that there is a very similar post to this one(Failed to locate the winutils binary in the hadoop binary path), however, I have tried every step that was suggested and the same error still appears.

I'm trying to use the Apache Spark version 1.6.0 on Windows 7 to perform the tutorial on this page http://spark.apache.org/docs/latest/streaming-programming-guide.html, specifically using this code:

./bin/run-example streaming.JavaNetworkWordCount localhost 9999

However, this error keeps appearing:

After reading this post Failed to locate the winutils binary in the hadoop binary path

I realized I needed the winutils.exe file, so I have downloaded a hadoop binary 2.6.0 with it, defined an Environment Variable called HADOOP_HOME:

 with value C:UsersGERALDesktophadoop-2.6.0in  

and placed it on Path like this: %HADOOP_HOME%

Yet the same error still appears when I try the code. Does anyone know how to solve this?

解决方案

If you are running Spark on Windows with Hadoop, then you need to ensure your windows hadoop installation is properly installed. to run spark you need to have winutils.exe and winutils.dll in your hadoop home directory bin folder.

I would ask you to try this first:

1) You can download .dll and .exe fils from the bundle in below link.

https://codeload.github.com/sardetushar/hadooponwindows/zip/master

2) Copy winutils.exe and winutils.dll from that folder to your $HADOOP_HOME/bin.

3) Set the HADOOP_HOME either in your spark-env.sh or at the command, and add HADOOP_HOME/bin to PATH.

and then try running.

If you need any assistance for hadoop installation help, there is a nice link, you can try it.

http://toodey.com/2015/08/10/hadoop-installation-on-windows-without-cygwin-in-10-mints/

But, that can wait. you can try the first few steps.

这篇关于Spark 1.6-未能在hadoop二进制路径中找到winutils二进制文件的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！