为什么我的PySpark程序挂起之间如下 [英] Why does my PySpark program hang in between as follows

问题描述

我写了一个程序PySpark，当我运行它，它打印大量的输出，然后突然用下面的语句挂在屏幕上

I have written a PySpark program and when I run it , it prints a lot of output and then suddenly hangs with the following statement on the screen

[Stage 11:===================================>                   (65 + 1) / 100]

它不会从这里都取得进展。难道该计划实际上已经结束，或在这里别的东西错了吗？

It doesn't progress from here at all. Is it that the program has actually ended or something else wrong here?

推荐答案

这时候，你用的是很常见的阿帕奇星火可能有几个问题或异常情况不属于表现在终端，但调试这个你可以阅读发生了什么，同时运行，但取决于在Apache的星火当前的安装（这就是为什么我不张贴链接），你可以找到更多的信息这个文档。 监测和Apache的星火记录

That is very common when you use Apache Spark there might be several problems or exceptions which aren't showed in the terminal, however to debug this you can read what happened while running, but depends on your Apache Spark current installation (that's why I don't post a link), you can find more information on this doc. Monitoring and Logging in Apache Spark

您必须寻找这个 WebUI中在说明列，您可以点击 +详情（获得您需要点击在工作您将调试），如果你坚持下去，你会发现越来越多的信息。

You have to search for this WebUI, in the Description Column you can click on + details (to get that you need to click the job you will debug), and if you keep going you will find more and more information.

这篇关于为什么我的PySpark程序挂起之间如下的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！