如何在本地调试spark应用程序？ [英] How can I debug spark application locally?

问题描述

我很新兴的火花，我只想逐步学习如何在本地调试火花应用程序？任何人都可以详细说明这样做所需的步骤？

我可以从命令行本地在火花网站上运行simpleApp，但我只需要遍历代码，看看它是如何工作的。任何帮助非常感谢。谢谢。

解决方案

启动Spark shell。这是直接从 Spark文档：

  ./ bin / spark-shell --master local [2] 
  

您还将看到Spark shell被称为 REPL 。这是学习Spark的最好方式。我花了80％的时间在Spark shell中，另外20％将代码翻译成我的应用程序。

I am very new to spark and I just would like to learn step by step how to debug a spark application locally? Could anyone please detail the steps needed to do this?

I can run the simpleApp on the spark website locally from the command line but I just need to step through the code and see how it works. Any help is very much appreciated. Thank you.

解决方案

Fire up the Spark shell. This is straight from the Spark documentation:

./bin/spark-shell --master local[2]

You will also see the Spark shell referred to as the REPL. It is by far the best way to learn Spark. I spend 80% of my time in the Spark shell and the other 20% translating the code into my application.

这篇关于如何在本地调试spark应用程序？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！