mongodb:如果不存在则插入 [英] mongodb: insert if not exists

问题描述

每天，我都会收到一堆文件(更新).我要做的是插入每个尚不存在的项目.

Every day, I receive a stock of documents (an update). What I want to do is insert each item that does not already exist.

• 我还希望跟踪我第一次插入它们以及上次在更新中看到它们的情况.
• 我不想有重复的文件.
• 我不想删除以前已保存但不在我的更新中的文档.
• 每天有95％(估计)的记录未修改.

我正在使用Python驱动程序(pymongo).

I am using the Python driver (pymongo).

我目前要做的是(伪代码):

What I currently do is (pseudo-code):

for each document in update:
      existing_document = collection.find_one(document)
      if not existing_document:
           document['insertion_date'] = now
      else:
           document = existing_document
      document['last_update_date'] = now
      my_collection.save(document)

我的问题是它非常慢(少于10万条记录需要40分钟，而我在更新中有数百万条记录). 我很确定有一些内置函数可以执行此操作，但是update()的文档是mmmhhh ....有点简洁....(

My problem is that it is very slow (40 mins for less than 100 000 records, and I have millions of them in the update). I am pretty sure there is something builtin for doing this, but the document for update() is mmmhhh.... a bit terse.... (http://www.mongodb.org/display/DOCS/Updating )

有人可以建议如何更快地做到这一点吗?

Can someone advise how to do it faster?

推荐答案

听起来像您想做的"upsert". MongoDB对此具有内置支持.将一个额外的参数传递给您的update()调用:{upsert:true}.例如:

Sounds like you want to do an "upsert". MongoDB has built-in support for this. Pass an extra parameter to your update() call: {upsert:true}. For example:

key = {'key':'value'}
data = {'key2':'value2', 'key3':'value3'};
coll.update(key, data, upsert=True); #In python upsert must be passed as a keyword argument

这将完全替换您的if-find-else-update块.如果密钥不存在，它将插入；如果密钥不存在，它将更新.

This replaces your if-find-else-update block entirely. It will insert if the key doesn't exist and will update if it does.

之前:

{"key":"value", "key2":"Ohai."}

之后:

{"key":"value", "key2":"value2", "key3":"value3"}

您还可以指定要写入的数据:

You can also specify what data you want to write:

data = {"$set":{"key2":"value2"}}

现在，您选择的文档将仅更新"key2"的值，而其他所有内容均保持不变.

Now your selected document will update the value of "key2" only and leave everything else untouched.

这篇关于mongodb:如果不存在则插入的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！