如何保留 YARN 的日志文件? [英] How to keep YARN's log files?
问题描述
突然，我的 YARN 集群停止工作，我提交的所有内容都失败并显示退出代码 1".我想追踪这个问题，但是一旦应用程序失败，YARN 就会删除日志文件.我必须为 YARN 调整哪些配置设置才能保留这些日志文件?
Suddenly, my YARN cluster has stopped working, everything I submit fails with "Exit code 1". I want to track down that problem, but as soon as an application failed, YARN deletes the log files. What is the configuration setting I have to adjust for YARN to keep these log files?
推荐答案
您的容器似乎正在退出，退出代码为 1.
It seems your container is exiting with exit code 1.
您无法在 UI 上看到日志，因为默认情况下，日志聚合处于禁用状态.以下参数确定日志聚合:yarn.log-aggregation-enable"(如果禁用日志聚合，则设置为false").
You are unable to see the logs on the UI, because by default, the log aggregation is disabled. Following parameter determines the log aggregation: "yarn.log-aggregation-enable" (set to "false" if log aggregation is disabled).
如果设置为false"，则所有节点管理器将容器日志存储在本地目录中，由以下配置参数确定:yarn.nodemanager.log-dirs".
If this is set to "false", then all the node managers store the container logs in a local directory, determined by the following configuration parameter: "yarn.nodemanager.log-dirs".
例如就我而言，这设置为:
For e.g. in my case, this is set to:
<property>
<name>yarn.nodemanager.log-dirs</name>
<value>e:hdpdatahadooplogs</value>
</property>
因此，特定应用程序的所有容器日志都将在节点管理器机器中的文件夹e:hdpdatahadooplogs{application-id}{container-id}"中找到，其中Application Master 运行了.
So, all my container logs for a particular application, will be found in the folder "e:hdpdatahadooplogs {application-id} {container-id}", in the Node Manager machine, where the Application Master ran.
让我们假设我的应用程序:application_1443377528298_0010"失败.在 YARNRM 的 UI(由配置参数确定:yarn.resourcemanager.webapp.address)中，您可以获得有关应用程序管理器运行所在节点的信息.在下图中，应用程序管理器在机器120243"上运行.
Let's assume that my application: "application_1443377528298_0010" FAILED. In the YARNRM's UI (determined by config parameter: yarn.resourcemanager.webapp.address), you can get the information about the node, on which the Application Manager ran. In the figure below, the Application Manager ran on the machine "120243".
如果您登录本机并在文件夹e:hdpdatahadooplogsapplication_1443377528298_0010"中搜索，您可以看到应用程序application_1443377528298_0010"的所有容器的日志.
If you login to this machine and search in the folder "e:hdpdatahadooplogsapplication_1443377528298_0010", you can see the logs for all the containers of application "application_1443377528298_0010".
但是，现在如果您想通过 YARN RM Web UI 查看日志，则需要启用日志聚合.为此，您需要在 yarn-site.xml 中设置以下参数:
But, now if you want to see the logs through YARN RM web UI, then you need to enable the log aggregation. For that, you need to set the following parameters, in yarn-site.xml:
<property>
<name>yarn.log-aggregation-enable</name>
<value>true</value>
</property>
<property>
<name>yarn.nodemanager.remote-app-log-dir</name>
<value>/app-logs</value>
</property>
<property>
<name>yarn.nodemanager.remote-app-log-dir-suffix</name>
<value>logs</value>
</property>
通过上述设置，我的日志在 HDFS 中聚合在/app-logs/{username}/logs/".在此文件夹下，您可以找到迄今为止运行的所有应用程序的日志.同样，日志保留时间由配置参数yarn.log-aggregation.retain-seconds"(聚合日志保留多长时间)决定.
With the above settings, my logs are aggregated in HDFS at "/app-logs/{username}/logs/". Under this folder, you can find logs for all the applications run so far. Again the log retention is determined by the configuration parameter "yarn.log-aggregation.retain-seconds" (how long to retain the aggregated logs).
当 MapReduce 应用程序运行时，您可以从 YARN 的 Web UI 访问日志.应用程序完成后，日志将通过 Job History Server 提供.
When the MapReduce applications are running, then you can access the logs from the YARN's web UI. Once the application is completed, the logs are served through Job History Server.
在您的情况下，如果您想在 Web UI 上查看日志，则在应用程序终止后，您还需要开始运行 MapReduce Job History 服务器.要启用它，请在 mapred-site.xml 中设置以下配置参数:
In your case, if you want to see the logs on the Web UI, after the application is terminated, then you need to start running the MapReduce Job History server also. To enable it, set following configuration parameters in mapred-site.xml:
<property>
<name>mapreduce.jobhistory.address</name>
<value>{job-history-hostname}:10020</value>
</property>
<property>
<name>mapreduce.jobhistory.webapp.address</name>
<value>{job-history-hostname}:19888</value>
</property>
并在yarn-site.xml中设置如下配置参数:
And set following configuration parameter in yarn-site.xml:
<property>
<name>yarn.log.server.url</name>
<value>http://{job-history-hostname}:19888/jobhistory/logs</value>
</property>
我从 Windows 上的 HDP 安装中复制了设置，这些设置对我有用.这些也应该对你有用.上面提到的每个配置的描述，请参考以下链接:
I have replicated settings from HDP installation on Windows and these settings work for me. These should work for you also. For the description of each of the configurations mentioned above, refer the links below:
https://hadoop.apache.org/docs/r2.4.1/hadoop-yarn/hadoop-yarn-common/yarn-default.xml
这篇关于如何保留 YARN 的日志文件?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
