Hive 中的 JOIN 触发 MapReduce 中的哪种类型的 JOIN? [英] JOIN in Hive triggers which type of JOIN in MapReduce?

问题描述

如果我在 hive 中有一个使用 JOIN 的查询，让我们说两个表 上的 LEFT OUTER JOIN 或 INNER JOIN>ON 任何列，那么我如何知道它在后端 MapReduce 中转换为哪种类型的 JOIN(即 Map-side JOIN 或 Reduce-side JOIN)?

If I have a query in hive which employs JOIN, lets say a LEFT OUTER JOIN or an INNER JOIN on two tables ON any column, then how do I know which type of JOIN is it getting converted into in the back-end MapReduce (i.e. Map-side JOIN or Reduce-side JOIN) ?

谢谢.

推荐答案

使用 explain select ... 并检查计划.它解释了 map 和 reduce 究竟会做什么.此外，在执行期间，您可以检查作业跟踪器上的日志并查看映射器或减速器进程正在做什么.

Use explain select ... and check the plan. It explains what exactly map and reduce will do. Also during execution you can check logs on job tracker and see what mapper or reducer processes are doing.

例如下面的解释计划说它是map-side join(注意计划中的Map Join Operator):

For example the following piece of explain plan says that it is map-side join (Note Map Join Operator in the plan):

 Stage: Stage-33
    Map Reduce
      Map Operator Tree:
          TableScan
            **alias: s**
            filterExpr: (col is not null) (type: boolean)
            Statistics: Num rows: 85 Data size: 78965 Basic stats: COMPLETE Column stats: NONE
            Filter Operator
              predicate: (col is not null) (type: boolean)
              Statistics: Num rows: 22 Data size: 20438 Basic stats: COMPLETE Column stats: NONE
              **Map Join Operator
                condition map:
                     Inner Join 0 to 1**

这篇关于Hive 中的 JOIN 触发 MapReduce 中的哪种类型的 JOIN?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！