如何从Hive表中删除重复的记录？ [英] How to delete duplicate records from Hive table?

问题描述

我的Hive表：'dynpart'与列：Id，Name，Technology

  Id名称技术
 1 Abcd Hadoop 
 2 Efgh Java 
 3 Ijkl MainFrames 
 2 Efgh Java 
  

我们在选择查询中使用了'Distinct'从表中检索数据。任何人都可以告诉如何使用删除查询从Hive表中删除重复的行。

确保不推荐使用删除/更新记录的标准蜂巢。但我想知道我们该怎么做。

解决方案

您可以使用insert overwrite语句来更新数据。 b
$ b

  insert overwrite table dynpart select distinct * from dynpart; 
  

I am trying to learn about deleting duplicate records from a Hive table.

My Hive table: 'dynpart' with columns: Id, Name, Technology

Id  Name  Technology
1   Abcd  Hadoop
2   Efgh  Java
3   Ijkl  MainFrames
2   Efgh  Java

We have options like 'Distinct' to use in a select query, but a select query just retrieves data from the table. Could anyone tell how to use a delete query to remove the duplicate rows from a Hive table.

Sure that it is not recommended or not the standard to Delete/Update records in Hive. But I want to learn how do we do it.

解决方案

You can use insert overwrite statement to update data

insert overwrite table dynpart select distinct * from dynpart;

这篇关于如何从Hive表中删除重复的记录？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！