'yarn application -list'没有显示任何结果 [英] 'yarn application -list' doesnt show any results

问题描述

我已经在YARN群集上运行了一些Spark应用程序.该应用程序显示在YARN UI http://host:8088/cluster 的所有应用程序"页面中 yarn application -list 命令不会给出任何结果.可能是什么原因造成的?

I have run some Spark applications on a YARN cluster. The application shows up in the "All applications" page in the YARN UI http://host:8088/cluster but the yarn application -list command doesnt give any results. What could be the cause of this ?

推荐答案

事实证明，我在YARN中启用了日志聚合，但是设置了 yarn.nodemanager.remote-app-log-dir 到自定义hdfs目录(/tmp/yarnlogs)，因此日志实际上是在HDFS的/tmp/yarnlogs上聚合的，但是yarn命令仍在HDFS的默认位置(/tmp/logs)搜索日志.因此，将属性更改为其默认值对我来说就是固定的.

It turns out that I had enabled Log aggregation in YARN but had set the yarn.nodemanager.remote-app-log-dir to a custom hdfs directory (/tmp/yarnlogs), So logs were actually getting aggregated at /tmp/yarnlogs in HDFS, but the yarn command was still searching for logs at the default location on HDFS (/tmp/logs). So changing the property to its default value fixed it for me.

注意: 如果日志聚合目录配置错误，则在尝试从Web UI访问作业历史记录时也会引起错误，如下所示:
Log aggregation has not completed or is not enabled

NOTE: If the log aggregation directory is misconfigured ,it also causes an a error while trying to access job history from the web UI, that looks like :
Log aggregation has not completed or is not enabled

这篇关于'yarn application -list'没有显示任何结果的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！