MSCK REPAIR配置单元外部表 [英] MSCK REPAIR hive external tables

问题描述

我每天都将数据摄取到HDFS中.从数据到HDFS，我生成按日期划分的Hive外部表.我的问题如下，我应该在每次摄取数据后运行MSCK REPAIR TABLE表名，在这种情况下，我必须每天运行命令.或者只在创建表时运行一次就足够了.非常感谢您的回答

I have a daily ingestion of data in to HDFS . From data into HDFS I generate Hive external tables partitioned by date . My qestion is as follows , should I run MSCK REPAIR TABLE tablename after each data ingestion , in this case I have to run the command each day. Or running it just one time at the table creation is enough . Thanks a lot for your answers

最好的问候

推荐答案

您只能运行 MSCK REPAIR TABLE ，而结构或分区外部表已更改.此命令将更新表的元数据.

You only run MSCK REPAIR TABLE while the structure or partition of the external table is changed. This command updates the metadata of the table.

一个经常发生的例子，例如

One example that usually happen, e.g.

您使用字段 dt 表示分区表的日期.

You use a field dt which represent a date to partition the table.

• 昨天，您插入了 dt = 2018-06-12 的一些数据，然后应运行 MSCK REPAIR TABLE 来更新元数据，以告诉蜂巢知道新分区 dt = 2018-06-12 .
• 今天，您插入一些 dt = 2018-06-13 的数据，然后应运行 MSCK REPAIR TABLE 来更新元数据，以告诉蜂巢知道新分区 dt = 2018-06-13 .

• Yesterday, you inserted some data which is dt=2018-06-12, then you should run MSCK REPAIR TABLE to update the metadata to tell hive to aware a new partition dt=2018-06-12.
• Today, you insert some data which is dt=2018-06-13, then you should run MSCK REPAIR TABLE to update the metadata to tell hive to aware a new partition dt=2018-06-13.

这篇关于MSCK REPAIR配置单元外部表的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！