如何星火升级到更新的版本？ [英] How to upgrade Spark to newer version?

问题描述

我有一个虚拟机星火1.3 就可以了，但我想将它升级到星火1.5 主要由于某些支持的功能而不是130。是否有可能，我可以从 1.3 到 1.5 星火版本C>，如果是的话那么我该怎么办呢？

I have a virtual machine which has Spark 1.3 on it but I want to upgrade it to Spark 1.5 primarily due certain supported functionalities which were not in 1.3. Is it possible I can upgrade the Spark version from 1.3 to 1.5 and if yes then how can I do that?

推荐答案

pre-建星火分布，像我相信你是基于<一个使用href=\"http://stackoverflow.com/questions/33902079/numpy-error-in-printing-a-rdd-in-spark-with-ipython\">another你的问题，都是相当简单的升级，因为星火实际上不是装。事实上，所有你需要做的是：

Pre-built Spark distributions, like the one I believe you are using based on another question of yours, are rather straightforward to "upgrade", since Spark is not actually "installed". Actually, all you have to do is:

• 下载适当星火发行（pre-内置Hadoop的2.6及更高版本，你的情况）


• 解压适当的目录tar文件（夹i.e.where 火花1.3.1彬hadoop2.6 已经是）


• 更新 SPARK_HOME （也可能根据您的设​​置其他一些环境变量）相应

• Download the appropriate Spark distro (pre-built for Hadoop 2.6 and later, in your case)
• Unzip the tar file in the appropriate directory (i.e.where folder spark-1.3.1-bin-hadoop2.6 already is)
• Update your SPARK_HOME (and possibly some other environmental variables depending on your setup) accordingly

下面是我只是做我自己，从1.3.1到1.5.2，在设置你类似的（无业游民VM运行Ubuntu）：

Here is what I just did myself, to go from 1.3.1 to 1.5.2, in a setting similar to yours (vagrant VM running Ubuntu):

1）下载tar文件中的相应目录

1) Download the tar file in the appropriate directory

vagrant@sparkvm2:~$ cd $SPARK_HOME
vagrant@sparkvm2:/usr/local/bin/spark-1.3.1-bin-hadoop2.6$ cd ..
vagrant@sparkvm2:/usr/local/bin$ ls
ipcluster     ipcontroller2  iptest   ipython2    spark-1.3.1-bin-hadoop2.6
ipcluster2    ipengine       iptest2  jsonschema
ipcontroller  ipengine2      ipython  pygmentize
vagrant@sparkvm2:/usr/local/bin$ sudo wget http://apache.tsl.gr/spark/spark-1.5.2/spark-1.5.2-bin-hadoop2.6.tgz
[...]
vagrant@sparkvm2:/usr/local/bin$ ls
ipcluster     ipcontroller2  iptest   ipython2    spark-1.3.1-bin-hadoop2.6
ipcluster2    ipengine       iptest2  jsonschema  spark-1.5.2-bin-hadoop2.6.tgz
ipcontroller  ipengine2      ipython  pygmentize

注意，确切的镜子，你应该用 wget的使用将可能比我的不同，这取决于您所在的位置;您将通过点击下载页面下载星火的链接得到这个，你选择后，封装类型进行下载。

Notice that the exact mirror you should use with wget will be probably different than mine, depending on your location; you will get this by clicking the "Download Spark" link in the download page, after you have selected the package type to download.

2）解压缩 TGZ 与文件

vagrant@sparkvm2:/usr/local/bin$ sudo tar -xzf spark-1.*.tgz
vagrant@sparkvm2:/usr/local/bin$ ls
ipcluster     ipcontroller2  iptest   ipython2    spark-1.3.1-bin-hadoop2.6
ipcluster2    ipengine       iptest2  jsonschema  spark-1.5.2-bin-hadoop2.6
ipcontroller  ipengine2      ipython  pygmentize  spark-1.5.2-bin-hadoop2.6.tgz

您可以看到，现在你有一个新的文件夹，火花1.5.2彬hadoop2.6 。

You can see that now you have a new folder, spark-1.5.2-bin-hadoop2.6.

3）更新相应 SPARK_HOME （也可能是您正在使用其他环境变量）来指向这个新的目录，而不是previous之一。

3) Update accordingly SPARK_HOME (and possibly other environmental variables you are using) to point to this new directory instead of the previous one.

和你应该做的，计算机重新启动后。

And you should be done, after restarting your machine.

注意：

1. 您不必删除previous星火分布，只要所有相关的环境变量指向新的。这样的话，你甚至可以快速移动回来，来回新旧版本之间，如果你要测试的东西（即你只需要改变相关的环境变量）。


2. 须藤在我的情况下，必要的;它可能是不必要根据您的设​​置。


3. 确保一切正常之后，这是好主意，删除下载 TGZ 文件。


4. 您可以使用完全相同的程序升级到星火未来的版本，因为他们来了（相当快）。如果你这样做，既确保previous TGZ 文件已被删除或修改焦油命令上述指向一个特定的文件（即没有 * 如上通配符）。

1. You don't need to remove the previous Spark distribution, as long as all the relevant environmental variables point to the new one. That way, you may even quickly move "back-and-forth" between the old and new version, in case you want to test things (i.e. you just have to change the relevant environmental variables).
2. sudo was necessary in my case; it may be unnecessary for you depending on your settings.
3. After ensuring that everything works fine, it's good idea to delete the downloaded tgz file.
4. You can use the exact same procedure to upgrade to future versions of Spark, as they come out (rather fast). If you do this, either make sure that previous tgz files have been deleted, or modify the tar command above to point to a specific file (i.e. no * wildcards as above).

这篇关于如何星火升级到更新的版本？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！