Spark 1.6-无法在hadoop二进制路径中找到winutils二进制文件 [英] Spark 1.6-Failed to locate the winutils binary in the hadoop binary path

问题描述

我知道这里有一个非常类似的帖子（

阅读本文后
失败在hadoop二进制路径中找到winutils二进制文件

我意识到我需要winutils.exe文件，所以我已经下载了一个hadoop二进制文件2.6.0它定义了一个名为HADOOP_HOME的环境变量：

 的值为C：\ Users \ GERAL\Desktop\hadoop- 2.6.0 \bin 
  

并将其放置在Path中，如下所示：％HADOOP_HOME％

然而，当我尝试代码时仍会出现同样的错误。有谁知道如何解决这个问题？

解决方案

如果您使用Hadoop在Windows上运行Spark，那么您需要确保您的Windows hadoop安装已正确安装。要运行spark，你需要在你的hadoop主目录的bin文件夹中有winutils.exe和winutils.dll。

1）您可以下载.dll和.exe文件从下面的链接中获取。

https：// codeload .github.com / sardetushar / hadooponwindows / zip / master
$ b 2）将winutils.exe和winutils.dll从该文件夹复制到$ HADOOP_HOME / bin 。

3）在spark-env.sh或命令中设置 HADOOP_HOME ，然后添加 HADOOP_HOME / bin 至 PATH 。

然后尝试运行。

如果您需要Hadoop安装帮助的帮助，您可以尝试一下。

http://toodey.com/2015/08/10/hadoop-installation-on-windows-without-cygwin-in-10-mints/

但是，那可以等待。您可以尝试前几个步骤。

I know that there is a very similar post to this one(Failed to locate the winutils binary in the hadoop binary path), however, I have tried every step that was suggested and the same error still appears.

I'm trying to use the Apache Spark version 1.6.0 on Windows 7 to perform the tutorial on this page http://spark.apache.org/docs/latest/streaming-programming-guide.html, specifically using this code:

./bin/run-example streaming.JavaNetworkWordCount localhost 9999

However, this error keeps appearing:

After reading this post Failed to locate the winutils binary in the hadoop binary path

I realized I needed the winutils.exe file, so I have downloaded a hadoop binary 2.6.0 with it, defined an Environment Variable called HADOOP_HOME:

 with value C:\Users\GERAL\Desktop\hadoop-2.6.0\bin  

and placed it on Path like this: %HADOOP_HOME%

Yet the same error still appears when I try the code. Does anyone know how to solve this?

解决方案

If you are running Spark on Windows with Hadoop, then you need to ensure your windows hadoop installation is properly installed. to run spark you need to have winutils.exe and winutils.dll in your hadoop home directory bin folder.

I would ask you to try this first:

1) You can download .dll and .exe fils from the bundle in below link.

https://codeload.github.com/sardetushar/hadooponwindows/zip/master

2) Copy winutils.exe and winutils.dll from that folder to your $HADOOP_HOME/bin.

3) Set the HADOOP_HOME either in your spark-env.sh or at the command, and add HADOOP_HOME/bin to PATH.

and then try running.

If you need any assistance for hadoop installation help, there is a nice link, you can try it.

http://toodey.com/2015/08/10/hadoop-installation-on-windows-without-cygwin-in-10-mints/

But, that can wait. you can try the first few steps.

这篇关于Spark 1.6-无法在hadoop二进制路径中找到winutils二进制文件的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！