配置单元tez执行错误 [英] Hive tez execution error

问题描述

我正在运行配置单元查询，并在设置hive.execution.engine = tez时遇到以下错误，而查询在engine = MR下工作。

I am running a hive query and I got the following error when setting the hive.execution.engine=tez, while the query is working under engine=MR.

FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.tez.TezTask

我的查询是一个内部连接，数据非常大。

My query is an inner join and the data is quite big.

另一件事是我以前遇到过这个问题。但tez后来工作，所以我认为这是关于蜂房不稳定的事情。

Another thing is that I have met this problem before. But tez works later so I thought it was about something unstable about hive.

推荐答案

通过配置单元运行HQL时，包含以下参数。这会给你详细的日志，你可以很容易地确定根本原因。

While running your HQL via hive include following parameter. This will give you detailed logs and you can determine the root cause easily.

-hiveconf hive.root.logger=DEBUG,console

我遇到了类似的问题，上面的属性帮助我度过了一段美好时光。

I faced similar problem and above property help me big time.

例如：我收到以下消息

16/04/14 10:29:26 ERROR exec.Task: Failed to execute tez graph.
org.apache.tez.dag.api.TezException: org.apache.hadoop.yarn.exceptions.InvalidResourceRequestException: Invalid resource request, requested memory < 0, or requested memory > max configured, requestedMemory=20480, maxMemory=11288

当我将设置更改为 11288 ，我的查询通过罚款。

When I changed my setting to 11288, my query went through fine.

希望这可以帮到你。

这篇关于配置单元tez执行错误的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！