pymongo 错误:bson.errors.InvalidBSON:“utf8"编解码器无法解码位置 25 中的字节 0xa1:起始字节无效 [英] pymongo error: bson.errors.InvalidBSON: 'utf8' codec can't decode byte 0xa1 in position 25: invalid start byte

问题描述

tasks = list(self.collection.find().sort('_id',pymongo.DESCENDING).limit(1000))

我在使用 pymongo 解决程序时遇到了麻烦.

I had a trouble when i use pymongo to solve a program.

File "D:\Python27\lib\site-packages\pymongo-3.2.1-py2.7-win-amd64.egg\pymongo\cursor.py", line 1097, in next文件D:\Python27\lib\site-packages\pymongo-3.2.1-py2.7-win-amd64.egg\pymongo\cursor.py"，第 1039 行，在 _refresh文件D:\Python27\lib\site-packages\pymongo-3.2.1-py2.7-win-amd64.egg\pymongo\cursor.py"，第 903 行，在 __send_message文件D:\Python27\lib\site-packages\pymongo-3.2.1-py2.7-win-amd64.egg\pymongo\helpers.py"，第 133 行，在 _unpack_responsebson.errors.InvalidBSON:'utf8' 编解码器无法解码位置 25 的字节 0xa1:无效起始字节

File "D:\Python27\lib\site-packages\pymongo-3.2.1-py2.7-win-amd64.egg\pymongo\cursor.py", line 1097, in next File "D:\Python27\lib\site-packages\pymongo-3.2.1-py2.7-win-amd64.egg\pymongo\cursor.py", line 1039, in _refresh File "D:\Python27\lib\site-packages\pymongo-3.2.1-py2.7-win-amd64.egg\pymongo\cursor.py", line 903, in __send_message File "D:\Python27\lib\site-packages\pymongo-3.2.1-py2.7-win-amd64.egg\pymongo\helpers.py", line 133, in _unpack_response bson.errors.InvalidBSON: 'utf8' codec can't decode byte 0xa1 in position 25: invalid start byte

tasks =self.collection.find().sort('_id',pymongo.DESCENDING).limit(1000)
for task in tasks:  #If i use this way,it will also touch this problem

task = self.collection.find_one()#It would do so,too

我进入pymongo找出原因.我发现问题可能是由以下代码引起的

I step into pymongo to find out the reason.I find that the problem maybe cause by follow codes

    result = {"cursor_id": struct.unpack("<q", response[4:12])[0],
          "starting_from": struct.unpack("<i", response[12:16])[0],
          "number_returned": struct.unpack("<i", response[16:20])[0],
          "data": bson.decode_all(response[20:], codec_options)}

在 pymongo helper.py 133 行在bson.decode_all中它显示了关于'oid'的解码失败的问题，'oid'是mongo中的_id.然后我复制文档并用新的_id制作一个相同的文档，然后我成功得到了文档.

in pymongo helper.py 133 line in bson.decode_all it show the problem cause by the failed decode about 'oid'，'oid' is the _id in mongo.Then I copy the document and make a same document with a new _id,then i success get the document .

如何解决for task in tasks:"样式的问题.

How can i solve the problem with the "for task in tasks:" style.

使用的pymongo版本:3.2.1

pymongo version used: 3.2.1

推荐答案

您需要将 unicode_decode_error_handler 参数传递给 MongoClient 并至少使用 pymongo 3.5.1.

You need to pass the unicode_decode_error_handler argument to MongoClient and use pymongo 3.5.1 at least.

import pymongo
import json
from pymongo import MongoClient

if __name__ == '__main__':

    client = MongoClient(
        host="whatever_your_host_is",
        maxPoolSize=50,
        unicode_decode_error_handler='ignore'
    )


    my_db=client['my_db']
    collection=my_db['my_collection']

    cursor = collection.find({"whatever": "some_stuff"})

    for document in cursor:
          print(document)

看起来忽略"在 Python 2.7 上是默认设置的，但在 Python 3.6.1 中你必须自己做.这将忽略 Unicode 错误并让游标继续迭代，pymongo 将尽力重建 JSON 数据.

Looks like that 'ignore' is set by default on Python 2.7, but in Python 3.6.1 you have to do it yourself. This will ignore the Unicode errors and let the cursor continue iterating, pymongo will try to do its best to reconstruct the JSON data.

这篇关于pymongo 错误:bson.errors.InvalidBSON:“utf8"编解码器无法解码位置 25 中的字节 0xa1:起始字节无效的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！