如何串联目录中的所有CSV，并使用Python将CSV名称添加为列 [英] How to concatenate all CSVs in a directory, adding CSV name as a column with Python

问题描述

• 我有一个包含约100 CSV(Downloads/challenges)的文件夹.
• 每个CSV具有相同的50多个列.
• 每个CSV的标题都类似于azerbaijan_challenge_entrants.csv.

• I have a folder with about 100 CSVs (Downloads/challenges).
• Each CSV has the same 50+ columns.
• Each CSV is titled something like azerbaijan_challenge_entrants.csv.

我想创建一个新的CSV(all_entrants.csv)，其中包括所有100个CSV的所有数据，并添加一列:challenge，其中应包括行数据来自的CSV的名称.

I want to create one new CSV (all_entrants.csv) that includes all data from all 100 CSVs, adding one new column: challenge, which should include the name of the CSV that the row data came from.

我通常喜欢Python这样的任务.但是我正在努力使这项工作.任何帮助将不胜感激！

I generally like Python for tasks like this. But I am struggling to make this work. Any help would be appreciated!

推荐答案

使用标准库和第三方库pandas中的os，这是可能的:

This is possible with os from the standard library and 3rd party library pandas:

import os
import pandas as pd

mypath = os.path.join('Downloads', 'challenges')

# get list of files
files = [f for f in os.listdir(mypath) if os.path.isfile(os.path.join(mypath, f))]

# build list of dataframes, adding "challenge" column
dfs = [pd.read_csv(os.path.join(mypath, f)).assign(challenge=f) for f in files]

# concatenate dataframes into one
df = pd.concat(dfs, ignore_index=True)

# write to csv
df.to_csv('all_entrants.csv')

这篇关于如何串联目录中的所有CSV，并使用Python将CSV名称添加为列的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！