蜂巢:修改外部表的位置花费的时间太长 [英] Hive: modify external table's location take too long

问题描述

Hive有两种表，分别是托管表和外部表，区别在于，您可以检查

Hive has two kinds of tables which are Managed and External Tables, for the difference, you can check Managed. VS External Tables.

当前，要将外部数据库从HDFS移到Alluxio，我需要将外部表的位置修改为alluxio://.

Currently, to move external database from HDFS to Alluxio, I need to modify external table's location to alluxio://.

该语句类似于:alter table catalog_page set location "alluxio://node1:19998/user/root/tpcds/1000/catalog_returns"

据我了解，这应该是一个简单的metastore修改，但是，对于某些表修改，将花费数十分钟.数据库本身包含大约1TB数据.

According to my understanding, it should be a simple metastore modification,however, for some tables modification, it will spend dozens of minutes. The database itself contains about 1TB data btw.

反正我有什么要加快表更改过程的速度吗?如果没有，为什么这么慢?欢迎发表任何评论，谢谢.

Is there anyway for me to accelerate the table alter process? If no, why it's so slow? Any comment is welcomed, thanks.

推荐答案

我在$HIVE_HOME/bin下找到了建议的方法metatool.

I found suggested way which is metatool under $HIVE_HOME/bin.

metatool -updateLocation <new-loc> <old-loc>      Update FS root location in the
                                          metastore to new location.Both
                                          new-loc and old-loc should be
                                          valid URIs with valid host names
                                          and schemes.When run with the
                                          dryRun option changes are
                                          displayed but are not persisted.
                                          When run with the
                                          serdepropKey/tablePropKey option
                                          updateLocation looks for the
                                          serde-prop-key/table-prop-key
                                          that is specified and updates
                                          its value if found.

使用此工具，位置修改非常快. (也许几秒钟.)

By using this tool, the location modification is very fast. (maybe several seconds.)

将此线程留给可能遇到相同情况的任何人使用.

Leave this thread here for anyone who might run into the same situation.

这篇关于蜂巢:修改外部表的位置花费的时间太长的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！