可以从文件加载 hiveconf 变量吗?(与 HiveQL 文件分开) [英] Can hiveconf variables be loaded from a file? (Separate from the HiveQL file)

问题描述

我经常有一大块 HiveQL，我想对某些变量使用不同的设置多次运行.

I often have a large block of HiveQL that I want to run multiple times with different settings for some variables.

一个简单的例子是:

set mindate='2015-01-01 00:00:00'
set maxdate='2015-04-01 00:00:00'
select * from my_table where the_date between ${hiveconf:mindate} and ${hiveconf:maxdate}

然后通过 hive -f myfile.sql > 运行myout.log

稍后，我想更改变量并重新运行.我还想要记录每次运行时变量的值.

Later, I would like to change the variables and re-run. I also want a record of what values the variables had each time I ran.

所以我目前制作了 HiveQL 文件的副本，除了变量值之外，这些副本是相同的.然而，这显然容易出错，因为如果我需要更改实际的 HiveQL，那么我必须在每个文件中更改它.

So I currently make copies of the HiveQL file that are the same except for the variable values. This is obviously error-prone, however, because if I need to change the actual HiveQL, then I have to change it in every file.

理想情况下，我可以将所有设置存储在 JSON 文件(或其他文件)中，并使我的 HiveQL 文件完全动态化.有没有办法做到这一点?

Ideally, I could store all my settings a JSON file (or whatever) and have my HiveQL file be totally dynamic. Is there any way to do this?

推荐答案

在配置文件中设置你的变量并在你的 hql 脚本中加载这个文件:

Set your variables in the config file and load this file in your hql script:

 source /path_to_your_config_file/config.hql; 

这篇关于可以从文件加载 hiveconf 变量吗?(与 HiveQL 文件分开)的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！