如何控制蜂巢作业名称但保留阶段信息? [英] How do I control a hive job name but keep the stage info?

问题描述

我的系统定期执行许多 hive 查询.当您查看作业跟踪器时，它们显示为SELECT field, other_field ..... (Stage-1)"等.这对我来说不是特别有帮助，所以我补充说:

set mapred.job.name = 更有帮助的名字；
到查询.现在我可以更好地区分它们了.但是，现在我被分成多个阶段的查询都显示为相同的名称.我最喜欢的是

set mapred.job.name = more有用的名称 (Stage-%d);
其中 %d 将被当前阶段编号替换.
这可能吗，有谁知道怎么做?

I have a number of hive queries that my system executes on a regular basis. When you look at the job tracker, they show up as "SELECT field, other_field ..... (Stage-1)" and similar. That's not particularly helpful to me, so I added:

set mapred.job.name = more helpful name;
to the query. Now I can tell them apart better. However, now my queries that get split into multiple stages all show up as the same name. What I'd ideally like is something along the lines of

set mapred.job.name = more helpful name (Stage-%d);
where the %d would get replaced by the current stage number.
Is this possible, and does anyone know how?

推荐答案

我不确定是否有一种方法可以完全实现您的愿望，但我可以提供其他方法.
除了使用 set mapred.job.name，您还可以在查询的开头添加注释，并使用 更有用的名称，如下所示:
--这是一个更有帮助的名字
SELECT 字段，other_field ....

I'm not sure there is a way to implement exactly what you wish but I can offer something else.
Instead of using set mapred.job.name you can add a comment in the beginning of the query with a more helpful name like this :
-- this is a more helpful name
SELECT field, other_field ....

然后，在 jobtracker 中，您将看到 -- 这是一个更有用的名称..... (Stage-%d)"

Then, in the jobtracker you'll see -- this is a more helpful name ..... (Stage-%d)"

这篇关于如何控制蜂巢作业名称但保留阶段信息?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！