删除 pandas 所有列中具有相同值的重复行 [英] Delete duplicate rows with the same value in all columns in pandas

问题描述

我有一个大约有50万行的数据框.如我所见，有很多重复的行，那么如何删除所有列(大约80列)中具有相同值的重复行，而不仅仅是一个?

I have a dataframe with about a half a million rows. As I could see, there are plenty of duplicate rows, so how can I drop duplicate rows that have the same value in all of the columns (about 80 columns), not just one?

df:

period_start_time    id    val1    val2    val3
06.13.2017 22:00:00  i53    32      2       10
06.13.2017 22:00:00  i32    32      2       10
06.13.2017 22:00:00  i32    4       2       8
06.13.2017 22:00:00  i32    4       2       8
06.13.2017 22:00:00  i32    4       2       8
06.13.2017 22:00:00  i20    7       7       22
06.13.2017 22:00:00  i20    7       7       22

所需的输出:

period_start_time    id    val1    val2    val3
06.13.2017 22:00:00  i53    32      2       10
06.13.2017 22:00:00  i32    32      2       10
06.13.2017 22:00:00  i32    4       2       8
06.13.2017 22:00:00  i20    7       7       22

推荐答案

使用 drop_duplicates :

df = df.drop_duplicates()
print (df)
     period_start_time   id  val1  val2  val3
0  06.13.2017 22:00:00  i53    32     2    10
1  06.13.2017 22:00:00  i32    32     2    10
2  06.13.2017 22:00:00  i32     4     2     8
5  06.13.2017 22:00:00  i20     7     7    22

这篇关于删除 pandas 所有列中具有相同值的重复行的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！