如何在 Windows 上设置 Spark? [英] How to set up Spark on Windows?

问题描述

我正在尝试在 Windows 上设置 Apache Spark.

搜索了一下，我明白我想要的是独立模式.为了在 Windows 中运行 Apache spark，我需要下载哪些二进制文件?我在 spark 下载页面看到了带有 hadoop 和 cdh 的发行版.

我在网络上没有对此的引用.对此的分步指南非常感谢.

解决方案

我发现在 Windows 上最简单的解决方案是从源代码构建.

您几乎可以遵循本指南:http://spark.apache.org/docs/latest/building-spark.html

下载并安装Maven，并将MAVEN_OPTS设置为指南中指定的值.

但是如果你只是在玩 Spark，并且实际上并不需要它在 Windows 上运行，因为你自己的机器正在运行 Windows，我强烈建议你在 Linux 虚拟机上安装 Spark.最简单的入门方法可能是下载由 Cloudera 或 Hortonworks 制作的现成镜像，然后使用捆绑版本的 Spark，或者从源代码或您可以从 spark 网站获得的已编译二进制文件安装自己的版本.

I am trying to setup Apache Spark on Windows.

After searching a bit, I understand that the standalone mode is what I want. Which binaries do I download in order to run Apache spark in windows? I see distributions with hadoop and cdh at the spark download page.

I don't have references in web to this. A step by step guide to this is highly appreciated.

解决方案

I found the easiest solution on Windows is to build from source.

You can pretty much follow this guide: http://spark.apache.org/docs/latest/building-spark.html

Download and install Maven, and set MAVEN_OPTS to the value specified in the guide.

But if you're just playing around with Spark, and don't actually need it to run on Windows for any other reason that your own machine is running Windows, I'd strongly suggest you install Spark on a linux virtual machine. The simplest way to get started probably is to download the ready-made images made by Cloudera or Hortonworks, and either use the bundled version of Spark, or install your own from source or the compiled binaries you can get from the spark website.

这篇关于如何在 Windows 上设置 Spark?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！