纱线星火执行人日志 [英] Spark executor logs on YARN

问题描述

我发动YARN客户端模式的分布式应用程序的Spark，一个Cloudera的集群上。过了一段时间我看到了Cloudera管理器中的一些错误。有些执行人断开连接时发生此系统。我想调试问题，但内部异常没有被YARN报道。

I'm launching a distributed Spark application in YARN client mode, on a Cloudera cluster. After some time I see some errors on Cloudera Manager. Some executors get disconnected and this happens systematically. I would like to debug the issue but the internal exception is not reported by YARN.

Exception from container-launch with container ID: container_1417503665765_0193_01_000003 and exit code: 1
ExitCodeException exitCode=1: 
    at org.apache.hadoop.util.Shell.runCommand(Shell.java:538)
    at org.apache.hadoop.util.Shell.run(Shell.java:455)
    at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:702)
    at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:196)
    at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:299)
    at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:81)
    at java.util.concurrent.FutureTask.run(FutureTask.java:262)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:745)

我怎么能看到异常的堆栈跟踪？看来只有应用程序异常退出的YARN报告。有没有办法看到火花执行人日志纱线配置？

How can I see the stacktrace of the exception? It seems that YARN reports only that the application exited abnormally. Is there a way to see spark executor log in YARN configuration ?

推荐答案

检查节点管理器的yarn.nodemanager.log-dir的属性。这是星火执行人容器运行时的日志位置。

Check NodeManager's yarn.nodemanager.log-dir property. It's the log location of when Spark executor container is running.

请注意，当应用程序完成节点管理器可以删除文件（日志聚合）。选中该文件详细。的http://hortonworks.com/blog/simplifying-user-logs-management-and-access-in-yarn/

Note that when the application finishes NodeManager may remove the files (Log Aggregation). Check this document for detail. http://hortonworks.com/blog/simplifying-user-logs-management-and-access-in-yarn/

这篇关于纱线星火执行人日志的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！