我可以将数据从一个配置单元分区移动到同一个表的另一个分区 [英] Can i move data from one hive partition to another partition of the same table

问题描述

我的分区基于年/月/日。每周使用SimpleDateFormat创建了一个错误的分区。使用日期格式中的YYYY将2017-31-12日期的数据移至2018-31-12。

  SimpleDateFormat sdf = new SimpleDateFormat（YYYY-MM-dd）; 
  

所以我想将我的数据从分区2018/12/31移到2017/12 /同一表中的31个。我没有找到任何相关的文档来做同样的事情。

解决方案

据我所知，您想将数据从2018-12-31分区至2017/12/31。以下是我对你如何做的解释。

 ＃从Hive / Beeline 
 ALTER TABLE TableName PARTITION（PartitionCol = 2018-12-31）RENAME TO PARTITION PartitionCol = 2017年12月31日）; 
  

FromSparkCode，您基本上必须启动hiveContext并从中运行相同的HQL。您可以参考我的答案此处如何启动hive上下文。

 ＃如果您想在HDFS级别执行操作，以下是
＃FromHive / beeline运行下面的HQL 
 ALTER TABLE TableName ADD IF NOT EXISTS PARTITION（PartitionCol = 2017-12-31）; 
 
＃现在来自HDFS只需将2018年的数据移至2017分区
 hdfs dfs -mv /your/table_hdfs/path/schema.db/tableName/PartitionCol=2018-12-31/ * /your/table_hdfs/path/schema.db/tableName/PartitionCol=2017-12-31/ 
 
＃如果您需要
 hdfs dfs -rm -r / your，请移除2018分区/table_hdfs/path/schema.db/tableName/PartitionCol=2018-12-31 
 
＃也可以从直线/配置单元
中删除alter table tableName drop如果存在分区（PartitionCol = 2018 -12-31）; 
 
＃最终修复表
 msck修复表tableName 
  

$ b

为什么我必须修复表格？

My partition is based on year/month/date. Using SimpleDateFormat for week year created a wrong partition . The data for the date 2017-31-12 was moved to 2018-31-12 using YYYY in the date format.

   SimpleDateFormat sdf = new SimpleDateFormat("YYYY-MM-dd");

So what I want is to move my data from partition 2018/12/31 to 2017/12/31 of the same table. I did not find any relevant documentation to do the same.

解决方案

From what I understood, you would like to move the data from 2018-12-31 partition to 2017/12/31. Below is my explanation of how you can do it.

#From Hive/Beeline
ALTER TABLE TableName PARTITION (PartitionCol=2018-12-31) RENAME TO PARTITION (PartitionCol=2017-12-31);

FromSparkCode, You basically have to initiate the hiveContext and run the same HQL from it. You can refer one my answer here on how to initiate the hive Context.

#If you want to do on HDFS level, below is one of the approaches
#FromHive/beeline run the below HQL
ALTER TABLE TableName ADD IF NOT EXISTS PARTITION (PartitionCol=2017-12-31);

#Now from HDFS Just move the data in 2018 to 2017 partition
hdfs dfs -mv /your/table_hdfs/path/schema.db/tableName/PartitionCol=2018-12-31/* /your/table_hdfs/path/schema.db/tableName/PartitionCol=2017-12-31/

#removing the 2018 partition if you require
hdfs dfs -rm -r /your/table_hdfs/path/schema.db/tableName/PartitionCol=2018-12-31

#You can also drop from beeline/hive
alter table tableName drop if exists partition (PartitionCol=2018-12-31);

#At the end repair the table
msck repair table tableName

Why do i have to repair the table ??

这篇关于我可以将数据从一个配置单元分区移动到同一个表的另一个分区的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！