在发送到 S3 之前，YARN 应用程序日志在哪里存储在 EMR 中 [英] Where does YARN application logs get stored in EMR before sending to S3

问题描述

我需要将 Yarn 应用程序日志从 EMR 写入 S3 以外的其他来源.. 你能告诉我应用程序日志在 EMR 主实例中的保存位置吗

I have a requirement to write Yarn application logs from EMR to different source other than S3 .. Can you please lep me where does applications logs get saved in EMR master instance

推荐答案

如果应用程序作为一个步骤提交给 emr，那么日志将驻留在:

If the application is submitted to the emr as a step then the logs will reside in:

/var/log/hadoop/steps/<<step-id>>/<<log-file>>

emr 的大部分日志都可以在主节点的 /var/logs 目录下找到

most logs for emr can be found under the /var/logs directory in the master node

您还可以使用 yarn cli 获取应用程序日志并将返回的日志流重定向到文件以执行任何您想要的操作.

you could also use the yarn cli to get the application logs and redirect the returned log stream to a file to do whatever you want with.

yarn logs -applicationId <<application_id>> > application_log_file.log 

这篇关于在发送到 S3 之前，YARN 应用程序日志在哪里存储在 EMR 中的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！