从CSV删除重复的行 [英] Remove duplicate rows from CSV

问题描述

我有一个看起来像这样的CSV文件

I have a CSV file that looks like this

red,75,right
red,344,right
green,3,center
yellow,3222,right
blue,9,center
black,123,left
white,68,right
green,47,left
purple,48,left
purple,988,right
pink,2677,left
white,34,right

我正在使用Python，并且正在尝试删除单元格1中重复的行.我知道我可以使用类似pandas的方法来实现此目的，但是我正在尝试使用标准的python CSV库来实现.

I am using Python and am trying to remove rows that have duplicate in cell 1. I know I can achieve this using something like pandas but I am trying to do it using standard python CSV library.

预期结果是...

red,75,right
green,3,center
yellow,3222,right
blue,9,center
black,123,left
white,68,right
purple,988,right
pink,2677,left

有人举个例子吗?

推荐答案

您可以简单地使用字典，其中颜色是键，值是行.如果颜色已在字典中，请忽略该颜色，否则将其添加并将该行写入新的csv文件中.

You can simply use a dictionary where the color is the key and the value is the row. Ignore the color if it is already in the dictionary, otherwise add it and write the row to a new csv file.

import csv

file_in = 'input_file.csv'
file_out = 'output_file.csv'
with open(file_in, 'rb') as fin, open(file_out, 'wb') as fout:
    reader = csv.reader(fin)
    writer = csv.writer(fout)
    d = {}
    for row in reader:
        color = row[0]
        if color not in d:
            d[color] = row  
            writer.writerow(row)
result = d.values()

result
# Output:
# [['blue', '9', 'center'],
# ['pink', '2677', 'left'],
# ['purple', '48', 'left'],
# ['yellow', '3222', 'right'],
# ['black', '123', 'left'],
# ['green', '3', 'center'],
# ['white', '68', 'right'],
# ['red', '75', 'right']]

以及csv文件的输出:

And the output of the csv file:

!cat output_file.csv
# Output:
# red,75,right
# green,3,center
# yellow,3222,right
# blue,9,center
# black,123,left
# white,68,right
# purple,48,left
# pink,2677,left

这篇关于从CSV删除重复的行的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！