如何使用java中的eclipse在Windows上本地运行Spark [英] How to run Spark locally on Windows using eclipse in java

问题描述

我正在尝试测试 Mllib 的 SVM 实现.我想使用 eclipse 在 Windows 上本地运行他们的 java 示例.我已经下载了为 Hadoop 2.6 预先构建的 Spark 1.3.1 .当我尝试运行示例代码时，我得到:

I'm trying to test Mllib's implementation of SVM. I want to run their java example locally on windows, using eclipse. I've downloaded Spark 1.3.1 pre-built for Hadoop 2.6 . When i try to run the example code, i get:

15/06/11 16:17:09 ERROR Shell: Failed to locate the winutils binary in the hadoop binary path
java.io.IOException: Could not locate executable null\bin\winutils.exe in the Hadoop binaries.

为了能够在此设置中运行示例代码，我应该更改什么?

What should i change in order to be able to run the example code in this setup?

推荐答案

1. 创建以下目录结构:C:\hadoop_home\bin"(或将C:\hadoop_home"替换为您喜欢的任何内容)

1. Create the following directory structure: "C:\hadoop_home\bin" (or replace "C:\hadoop_home" with whatever you like)

下载以下文件:http://public-repo-1.hortonworks.com/hdp-win-alpha/winutils.exe

将第 2 步中的文件放入第 1 步中的bin"目录.

Put the file from step 2 into the "bin" directory from step 1.

将hadoop.home.dir"系统属性设置为C:\hadoop_home"(或您在第 1 步中创建的任何目录，末尾不带\bin").

Set the "hadoop.home.dir" system property to "C:\hadoop_home" (or whatever directory you created in step 1, without the "\bin" at the end).

这篇关于如何使用java中的eclipse在Windows上本地运行Spark的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！