_csv.Error：字段大于字段limit（131072） [英] _csv.Error: field larger than field limit (131072)

问题描述

我有一个脚本在csv文件中读取非常大的字段：

I have a script reading in a csv file with very huge fields:

# example from http://docs.python.org/3.3/library/csv.html?highlight=csv%20dictreader#examples
import csv
with open('some.csv', newline='') as f:
    reader = csv.reader(f)
    for row in reader:
        print(row)

$ b b

但是，这会在一些csv文件上引发以下错误：

However, this throws the following error on some csv files:

_csv.Error: field larger than field limit (131072)

推荐答案

csv文件可能包含非常大的字段，因此增加 field_size_limit ：

The csv file might contain very huge fields, therefore increase the field_size_limit:

import sys
import csv

csv.field_size_limit(sys.maxsize)

sys.maxsize 适用于Python 2.x和3.x. sys.maxint 只适用于Python 2.x（ SO：what-is-sys-maxint-in-python-3 ）

sys.maxsize works for Python 2.x and 3.x. sys.maxint would only work with Python 2.x (SO: what-is-sys-maxint-in-python-3)

Geoff指出，上面的代码可能会导致以下错误： OverflowError：Python int太大，无法转换为C long 。
为了避免这种情况，您可以使用以下快速和脏的代码：

As Geoff pointed out, the code above might result in the following error: OverflowError: Python int too large to convert to C long. To circumvent this, you could use the following quick and dirty code:

import sys
import csv
maxInt = sys.maxsize
decrement = True

while decrement:
    # decrease the maxInt value by factor 10 
    # as long as the OverflowError occurs.

    decrement = False
    try:
        csv.field_size_limit(maxInt)
    except OverflowError:
        maxInt = int(maxInt/10)
        decrement = True

这篇关于_csv.Error：字段大于字段limit（131072）的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！