Google App Engine ndb.delete_multi()的效率如何? [英] How efficient is Google App Engine ndb.delete_multi()?

问题描述

我正在努力清除约10,000个实体的数据库，我的计划是将其放入一个任务中，该任务使用ndb.delete_multi()一次删除200个，然后再次递归调用自身，直到没有剩余实体为止

I'm working on something to clear my database of ~10,000 entities, and my plan is to put it in a task that deletes 200 at a time using ndb.delete_multi() and then recursively calls itself again until there are no entities left.

现在，我还没有递归，所以我可以手动运行几次代码，并检查错误，配额使用等.代码是:

For now, I don't have the recursion in it yet so I could run the code a few times manually and check for errors, quota use, etc. The code is:

entities = MyModel.query_all(ndb.Key('MyModel', '*defaultMyModel')).fetch(200)
key_list = ndb.put_multi(entities)
ndb.delete_multi(key_list)

query_all()所做的只是查询MyModel并返回所有内容.

All the query_all() does is query MyModel and return everything.

我已经通过注释掉东西并运行该方法进行了一些测试，看起来前两行占用了预期的写入量(〜200).

I've done some testing by commenting out things and running the method, and it looks like the first two lines take up the expected amount of writes (~200).

运行第三行ndb.delete_multi()约占我每天50,000个写配额的8％，因此大约4000次写-是我认为应该做的20倍.

Running the third line, ndb.delete_multi(), takes up about 8% of my 50,000 daily write allowance, so about 4000 writes--20 times as many as I think it should be doing.

我还确保key_list仅包含200个带有日志记录的键.

I've also made sure the key_list contains only 200 keys with logging.

关于为什么要占用这么多写的任何想法?我使用的方法是否错误?还是只使用大量的内存?在那种情况下，我有什么办法可以更有效地做到这一点?

Any ideas on why this takes up so many writes? Am I using the method wrong? Or does it just use a ton of memory? In that case, is there any way for me to do this more efficiently?

谢谢.

推荐答案

您的代码示例效率极低.如果要删除大量实体，则需要对以下实体进行批处理，但是，应使用keys_only查询检索数据，然后删除:

Your code example is extremely inefficient. If you are deleting large numbers of entities than you will need to batch the below but, you should be retrieving data with a keys_only query and then deleting:

from google.appengine.ext import ndb

ndb.delete_multi(
    MyModel.query().fetch(keys_only=True)
)

关于写操作的数量(请参阅Andrei的答案)，请仅确保模型上需要索引的字段已启用索引".

In regards to the number of write operations (see Andrei's answer), ensure only the fields on your model that are required to be indexed "have an index enabled".

这篇关于Google App Engine ndb.delete_multi()的效率如何?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！