python csv unicode'ascii'编解码器不能编码字符u'\xf6'在位置1：序数不在范围（128） [英] python csv unicode 'ascii' codec can't encode character u'\xf6' in position 1: ordinal not in range(128)

问题描述

我从[python网站] [1]复制了这个脚本这是另一个问题，但现在的编码问题：

I have copied this script from [python web site][1] This is another question but now problem with encoding:

import sqlite3
import csv
import codecs
import cStringIO
import sys

class UTF8Recoder:
    """
    Iterator that reads an encoded stream and reencodes the input to UTF-8
    """
    def __init__(self, f, encoding):
        self.reader = codecs.getreader(encoding)(f)

    def __iter__(self):
        return self

    def next(self):
        return self.reader.next().encode("utf-8")

class UnicodeReader:
    """
    A CSV reader which will iterate over lines in the CSV file "f",
    which is encoded in the given encoding.
    """

    def __init__(self, f, dialect=csv.excel, encoding="utf-8", **kwds):
        f = UTF8Recoder(f, encoding)
        self.reader = csv.reader(f, dialect=dialect, **kwds)

    def next(self):
        row = self.reader.next()
        return [unicode(s, "utf-8") for s in row]

    def __iter__(self):
        return self

class UnicodeWriter:
    """
    A CSV writer which will write rows to CSV file "f",
    which is encoded in the given encoding.
    """

    def __init__(self, f, dialect=csv.excel, encoding="utf-8", **kwds):
        # Redirect output to a queue
        self.queue = cStringIO.StringIO()
        self.writer = csv.writer(self.queue, dialect=dialect, **kwds)
        self.stream = f
        self.encoder = codecs.getincrementalencoder(encoding)()

    def writerow(self, row):
        self.writer.writerow([s.encode("utf-8") for s in row])
        # Fetch UTF-8 output from the queue ...
        data = self.queue.getvalue()
        data = data.decode("utf-8")
        # ... and reencode it into the target encoding
        data = self.encoder.encode(data)
        # write to the target stream
        self.stream.write(data)
        # empty queue
        self.queue.truncate(0)

    def writerows(self, rows):
        for row in rows:
            self.writerow(row)

这次编码的问题，当我运行它给我这个错误：

This time problem with encoding, when I ran this it gave me this error:

Traceback (most recent call last):
  File "makeCSV.py", line 87, in <module>
    uW.writerow(d)
  File "makeCSV.py", line 54, in writerow
    self.writer.writerow([s.encode("utf-8") for s in row])
AttributeError: 'int' object has no attribute 'encode'

将所有整数转换为字符串，但这次我得到了这个错误：

Then I converted all integers to string, but this time I got this error:

Traceback (most recent call last):
  File "makeCSV.py", line 87, in <module>
    uW.writerow(d)
  File "makeCSV.py", line 54, in writerow
    self.writer.writerow([str(s).encode("utf-8") for s in row])
UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 1: ordinal not in range(128)

我已经实现上面的处理unicode字符，但它给我这样的错误。

I have implemented above to deal with unicode characters, but it gives me such error. What is the problem and how to fix it?

推荐答案

然后我将所有的整数转换为字符串，

Then I converted all integers to string,

您将整数和字符串转换为字节字符串。对于字符串，这将使用默认的字符编码，恰好是ASCII，并且当您有非ASCII字符时，这将失败。您要 unicode 而不是 str 。

You converted both integers and strings to byte strings. For strings this will use the default character encoding which happens to be ASCII, and this fails when you have non-ASCII characters. You want unicode instead of str.

self.writer.writerow([unicode(s).encode("utf-8") for s in row])

在调用该方法之前将所有内容转换为unicode可能更好。该类专门用于解析Unicode字符串。它不是为支持其他数据类型而设计的。

It might be better to convert everything to unicode before calling that method. The class is designed specifically for parsing Unicode strings. It was not designed to support other data types.

这篇关于python csv unicode'ascii'编解码器不能编码字符u'\xf6'在位置1：序数不在范围（128）的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！