为什么我的纱线应用程序即使启用了日志记录也没有日志？ [英] Why does my yarn application not have logs even with logging enabled?

问题描述

我已经在xml文件中启用了日志： yarn-site.xml ，并且我重新开始 yarn ：



  sudo service hadoop-yarn-resourcemanager restart 
 sudo服务hadoop-yarn-nodemanager restart 
  

我运行了我的应用程序，然后在<应用程序ID>中找到 applicationID code> yarn application -list 。所以，我这样做： yarn logs -applicationId< application ID> ，我得到以下结果：

 hdfs：//< ip address> / var / log / hadoop-yarn / path / to / application /没有任何日志文件
  

我是否需要更改其他配置？还是我以错误的方式访问日志？

谢谢。

解决方案

  yarn application -list 
  

将仅列出位于 SUBMITTED ， ACCEPTED 或 RUNNING 状态。

日志聚合收集每个容器的日志，并将这些日志移动到 yarn.nodemanager.remote-app-log-dir 只能在应用程序完成之后。请参阅 yarn.log-aggregation-enable 属性的说明 here 。 因此， applicationId 尚未完成，并且日志尚未收集。因此，尝试访问正在运行的应用程序的日志时的响应



  hdfs：//< ip address> / var / log / hadoop-yarn / path / to / application /没有任何日志文件
  

您可以尝试相同的命令 yarn logs -applicationId<应用程序ID> 可在应用程序完成后查看日志。

列出所有 FINISHED 应用程序，使用

  yarn application -list  - appStates FINISHED 
  

或列出所有应用程序

  yarn application -list -appStates ALL 
  

I have enabled logs in the xml file: yarn-site.xml, and I restarted yarn by doing:

sudo service hadoop-yarn-resourcemanager restart
sudo service hadoop-yarn-nodemanager restart

I ran my application, and then I see the applicationID in yarn application -list. So, I do this: yarn logs -applicationId <application ID>, and I get the following:

hdfs://<ip address>/var/log/hadoop-yarn/path/to/application/  does not have any log files

Do I need to change some other configuration? Or am I accessing the logs the wrong way?

Thank you.

解决方案

yarn application -list

will list only the applications that are either in SUBMITTED, ACCEPTED or RUNNING state.

Log aggregation collects each container's logs and moves these logs onto the directory configured in yarn.nodemanager.remote-app-log-dir only after the completion of the application. Refer the description of yarn.log-aggregation-enable property here.

So, the applicationId listed by the command isn't completed yet and the logs are not yet collected. Thus the response when trying to access the logs of a running application

hdfs://<ip address>/var/log/hadoop-yarn/path/to/application/  does not have any log files

You can try the same command yarn logs -applicationId <application ID> to view the logs once the application has completed.

To list all the FINISHED applications, use

yarn application -list -appStates FINISHED

Or to list all the applications

yarn application -list -appStates ALL

这篇关于为什么我的纱线应用程序即使启用了日志记录也没有日志？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！