可以从文件加载hiveconf变量吗？ （与HiveQL文件分开） [英] Can hiveconf variables be loaded from a file? (Separate from the HiveQL file)

问题描述

一个简单的例子是：

  set mindate ='2015-01-01 00:00:00'
 set maxdate ='2015-04-01 00： 00:00'
 select * from my_table where $ {hiveconf：mindate}和$ {hiveconf：maxdate} 
  

之间的日期

然后通过 hive -f myfile.sql>运行。 myout.log

稍后，我想更改变量并重新运行。我还想要一个记录，这些变量每次运行时都有什么值。

因此，我现在制作HiveQL文件的副本除了变量值相同外。然而，这显然很容易出错，因为如果我需要更改实际的HiveQL，那么我必须在每个文件中更改它。

理想情况下，我可以将所有我的设置一个JSON文件（或其他），并让我的HiveQL文件是完全动态的。有没有办法做到这一点？

解决方案

在配置文件中设置你的变量并在你的hql脚本中加载这个文件： / p>

  source /path_to_your_config_file/config.hql; 
  

I often have a large block of HiveQL that I want to run multiple times with different settings for some variables.

A simple example would be:

set mindate='2015-01-01 00:00:00'
set maxdate='2015-04-01 00:00:00'
select * from my_table where the_date between ${hiveconf:mindate} and ${hiveconf:maxdate}

Which is then run via hive -f myfile.sql > myout.log

Later, I would like to change the variables and re-run. I also want a record of what values the variables had each time I ran.

So I currently make copies of the HiveQL file that are the same except for the variable values. This is obviously error-prone, however, because if I need to change the actual HiveQL, then I have to change it in every file.

Ideally, I could store all my settings a JSON file (or whatever) and have my HiveQL file be totally dynamic. Is there any way to do this?

解决方案

Set your variables in the config file and load this file in your hql script:

 source /path_to_your_config_file/config.hql; 

这篇关于可以从文件加载hiveconf变量吗？ （与HiveQL文件分开）的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！