pandas ,读CSV时会忽略多余的逗号 [英] Pandas, read CSV ignoring extra commas
问题描述
我正在将具有8列的CSV文件读取到Pandas数据框中.最后一列包含错误消息,其中一些包含逗号.这将导致文件读取失败,并显示错误ParserError: Error tokenizing data. C error: Expected 8 fields in line 21922, saw 9
I am reading a CSV file with 8 columns into Pandas data frame. The final column contains an error message, some of which contain commas. This causes the file read to fail with the error ParserError: Error tokenizing data. C error: Expected 8 fields in line 21922, saw 9
是否有一种方法可以忽略第8个字段之后的所有逗号,而不必遍历文件并删除多余的逗号?
Is there a way to ignore all commas after the 8th field, rather than having to go through the file and remove excess commas?
读取文件的代码:
import pandas as pd
df = pd.read_csv('C:\\somepath\\output.csv')
行有效:
061AE,Active,001,2017_02_24 15_18_01,00006,1,00013,some message
行失败:
061AE,Active,001,2017_02_24 15_18_01,00006,1,00013,longer message, with commas
推荐答案
您可以使用re.sub
将前几个逗号替换为'|',然后将中间结果保存在StringIO
中,然后对其进行处理
You can use re.sub
to replace the first few commas with, say, the '|', save the intermediate results in a StringIO
then process that.
import pandas as pd
from io import StringIO
import re
for_pd = StringIO()
with open('MikeS159.csv') as mike:
for line in mike:
new_line = re.sub(r',', '|', line.rstrip(), count=7)
print (new_line, file=for_pd)
for_pd.seek(0)
df = pd.read_csv(for_pd, sep='|', header=None)
print (df)
我将您问题的两行放入文件中以获取此输出.
I put the two lines from your question into a file to get this output.
0 1 2 3 4 5 6 \
0 061AE Active 1 2017_02_24 15_18_01 6 1 13
1 061AE Active 1 2017_02_24 15_18_01 6 1 13
7
0 some message
1 longer message, with commas
这篇关于 pandas ,读CSV时会忽略多余的逗号的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!