在Windows上安装Apache星火 [英] Installing Apache Spark on Windows

问题描述

我需要安装阿帕奇星火在Windows机器上。根据该文件，我应该已经安装了SBT我的机器上，并覆盖其默认选项，最多使用2G内存的。

想了很多，我决定选择行家，而不是之后（我overrided默认选项，使用512MB），我可以运行 MVN编译即开始下载一些依赖，直到它达到锌服务器。它说didn`t端口8080和Maven运行锌服务器无法构建。

我是否需要安装在本地服务器锌，并将其添加到Windows路径？我可以编译我的项目，而不服务器？

解决方案

我刚开始学习Apache的火花，所以我可能从有效的答案远。它为我工作在Mac OS X让你millage可能会有所不同。

构建过程中所描述的大厦：

  

星火使用简单生成工具，这是捆绑在一起的。要编译
  code，进入顶级星火目录并运行

  SBT / SBT总成
 

我没有更改默认构建设置做组装。

从消息人士处昨日的构建：

 亚切克：〜/ OSS /火花
$ SBT /舒巴坦总成
...
[成功]总时间：3632 S，完成了2014年5月6日上午02点39分07秒
 

从一些环境变量，很少有其他的模块添加到总装源今天的构建：

 亚切克：〜/ OSS /火花
$ SPARK_HADOOP_VERSION = 2.4.0 SPARK_YARN =真SPARK_HIVE =真SBT /舒巴坦总成
...
[成功]总时间：4564 S，完成了2014年5月7日上午1点49分56秒
 

I need to install Apache Spark on a Windows machine. According to the documentation I should have sbt installed on my machine and also override its default options to use a maximum of 2G of RAM.

After trying a lot I decided to choose maven instead (I overrided the default options to use 512MB) and I could run mvn compile that started to download some dependencies until it reached Zinc server. It said Zinc server didn`t run on port 8080 and maven failed the build.

Do I need to install the Zinc server locally and add it to Windows path? Can I compile my project without the server?

解决方案

I've just started learning Apache Spark so I might be far from the valid answer. It worked for me on Mac OS X so your millage may vary.

The build process is described in Building:

Spark uses Simple Build Tool, which is bundled with it. To compile the code, go into the top-level Spark directory and run

sbt/sbt assembly

I've not changed the default build settings to do the assembly.

Yesterday's build from the sources:

jacek:~/oss/spark
$ sbt/sbt assembly
...
[success] Total time: 3632 s, completed May 6, 2014 2:39:07 AM

Today's build from the sources with some environment variables that add few other modules to the final assembly:

jacek:~/oss/spark
$ SPARK_HADOOP_VERSION=2.4.0 SPARK_YARN=true SPARK_HIVE=true sbt/sbt assembly
...
[success] Total time: 4564 s, completed May 7, 2014 1:49:56 AM

这篇关于在Windows上安装Apache星火的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！