即使批量记录中的一条失败，如何使用MongoDB批量插入（使用PyMongo）？ [英] How can I bulk insert with MongoDB (using PyMongo), even when one record of the bulk fails?

问题描述

我有一些 Python 代码，它们使用 PyMongo 将许多列表（每个列表包含1000个对象）插入具有唯一索引的集合中（字段名称为 data_id ）。

I have some Python code that uses PyMongo to insert many lists (of 1000 objects each), into a collection with a unique index (field name is data_id).

但是，我的某些对象列表在要插入的不同列表集中有重复的数据（例如，也许是第二个1000个对象具有一个或两个记录，这些记录与先前在第一批大容量插入中插入的某些对象相同。

However, some of my lists of objects have duplicate data in the different sets of lists to be inserted (e.g., perhaps the second list of 1000 objects has one or two records that are identical to some of the objects previously inserted in the first set of the bulk insert).

这是问题所在：当代码批量处理时，插入一组1000个对象，而一个对象先前已插入 data_id ，则所有1000个对象的整个插入操作均失败。我正在执行如下插入操作：

Here's the problem: when the code goes to bulk insert a set of 1000 objects, and one object has a previously inserted data_id, the entire insert for all 1000 object fails. I am performing the insert as below:

inserted = False
try:
    collection = self.db[self.database][self.collection]
    collection.insert(record)
    inserted = True

except pymongo.errors.ConnectionFailure, e:
    sys.stdout.write('Error connecting to %s: %s\n' % (self.connection_url, e))
except BaseException, e:
    sys.stdout.write('An error occurred in add_record: %s\n' % e)

return inserted

我已经在某处阅读（现在在任何地方都找不到参考！），可以通过告诉Mongo列表无序来避免这种情况。所以我尝试通过插入行 ordered = False ，但这失败了：

I have read somewhere (and now I can't find the reference anywhere!), that this can be avoided by telling Mongo the list is unordered. So I tried passing the insert line ordered=False, but this fails with:

__init__() got an unexpected keyword argument 'ordered'

有人知道如何使用 PyMongo.insert（）插入对象列表，以便仅非唯一记录失败，其余部分按预期插入？

Does anyone know how to use PyMongo.insert() to unordered insert a list of objects so that only the non-unique records fail and the rest are inserted as expected?

推荐答案

找到了答案。对于那些感兴趣的人， .insert（）已在 PyMongo 中弃用，建议使用 .insert_many（），它尊重 ordered = False 关键字。

Found the answer. For those interested, .insert() has been deprecated in PyMongo and it is advised to use .insert_many(), which respects the ordered=False keyword.

这篇关于即使批量记录中的一条失败，如何使用MongoDB批量插入（使用PyMongo）？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！