在纱簇模式下在YARN上运行Spark:控制台输出转到何处? [英] Running Spark on YARN in yarn-cluster mode: Where does the console output go?

问题描述

我关注了此页面，并在纱线群集模式下在YARN上运行了SparkPi示例应用程序.

I followed this page and ran the SparkPi example application on YARN in yarn-cluster mode.

http://spark.apache.org/docs/latest/running-on-yarn .html

最后我看不到程序的输出(在这种情况下，这是计算的结果).当我以yarn-client模式(--master yarn-client)运行它时，我看到这样的输出:

I don't see the output of the program at the end (which is the result of the computation in this case). When I run it in the yarn-client mode (--master yarn-client), I see an output like this:

Pi大致为3.138796

Pi is roughly 3.138796

标准输出在纱线簇模式下在哪里?

Where does the standard out go in the yarn-cluster mode?

推荐答案

经过一番摸索，我在

检查输出(将$ YARN_APP_ID替换为上一条命令的应用程序标识符"输出) (注意:根据Hadoop版本，YARN_APP_LOGS_DIR通常为/tmp/logs或$ HADOOP_HOME/logs/userlogs.)

Examine the output (replace $YARN_APP_ID in the following with the "application identifier" output by the previous command) (Note: YARN_APP_LOGS_DIR is usually /tmp/logs or $HADOOP_HOME/logs/userlogs depending on the Hadoop version.)

$ cat $ YARN_APP_LOGS_DIR/$ YARN_APP_ID/容器* _000001/stdout

$ cat $YARN_APP_LOGS_DIR/$YARN_APP_ID/container*_000001/stdout

Pi大致为3.13794

Pi is roughly 3.13794

我希望他们将此说明放在 1.1.0文档中也是.

I wish they put this instruction in the 1.1.0 documentation too.

这篇关于在纱簇模式下在YARN上运行Spark:控制台输出转到何处?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！