使用 Pandas 处理可变数量的列 - Python [英] Handling Variable Number of Columns with Pandas - Python

问题描述

我有一个看起来像这样的数据集(最多 5 列 - 但可以更少)

1,2,31,2,3,41、2、3、4、51,21,2,3,4....

我正在尝试使用 pandas read_table 将其读入 5 列数据框.我想在没有额外按摩的情况下阅读这篇文章.

如果我尝试

将pandas导入为pdmy_cols=['A','B','C','D','E']my_df=pd.read_table(path,sep=',',header=None,names=my_cols)

我收到一个错误 - 列名有 5 个字段，数据有 3 个字段".

有什么办法可以让pandas在读取数据时为缺失的列填充NaN?

解决方案

一种似乎有效的方法(至少在 0.10.1 和 0.11.0.dev-fc8de6d 中):

<预><代码>>>>!cat ragged.csv1,2,31,2,3,41、2、3、4、51,21,2,3,4>>>my_cols = ["A", "B", "C", "D", "E"]>>>pd.read_csv("ragged.csv", names=my_cols, engine='python')A B C D E0 1 2 3 NaN NaN1 1 2 3 4 南2 1 2 3 4 53 1 2 NaN NaN NaN4 1 2 3 4 南

请注意，此方法要求您为所需的列指定名称.不像其他一些方法那么通用，但在适用时效果很好.

I have a data set that looks like this (at most 5 columns - but can be less)

1,2,3
1,2,3,4
1,2,3,4,5
1,2
1,2,3,4
....

I am trying to use pandas read_table to read this into a 5 column data frame. I would like to read this in without additional massaging.

If I try

import pandas as pd
my_cols=['A','B','C','D','E']
my_df=pd.read_table(path,sep=',',header=None,names=my_cols)

I get an error - "column names have 5 fields, data has 3 fields".

Is there any way to make pandas fill in NaN for the missing columns while reading the data?

解决方案

One way which seems to work (at least in 0.10.1 and 0.11.0.dev-fc8de6d):

>>> !cat ragged.csv
1,2,3
1,2,3,4
1,2,3,4,5
1,2
1,2,3,4
>>> my_cols = ["A", "B", "C", "D", "E"]
>>> pd.read_csv("ragged.csv", names=my_cols, engine='python')
   A  B   C   D   E
0  1  2   3 NaN NaN
1  1  2   3   4 NaN
2  1  2   3   4   5
3  1  2 NaN NaN NaN
4  1  2   3   4 NaN

Note that this approach requires that you give names to the columns you want, though. Not as general as some other ways, but works well enough when it applies.

这篇关于使用 Pandas 处理可变数量的列 - Python的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！