Google App Engine ndb.delete_multi() 的效率如何? [英] How efficient is Google App Engine ndb.delete_multi()?

问题描述

我正在做一些事情来清除我的约 10,000 个实体的数据库，我的计划是将它放入一个任务中，使用 ndb.delete_multi() 一次删除 200 个，然后递归再次调用自身，直到没有实体为止.

I'm working on something to clear my database of ~10,000 entities, and my plan is to put it in a task that deletes 200 at a time using ndb.delete_multi() and then recursively calls itself again until there are no entities left.

目前，我还没有递归，所以我可以手动运行几次代码并检查错误、配额使用等.代码是:

For now, I don't have the recursion in it yet so I could run the code a few times manually and check for errors, quota use, etc. The code is:

entities = MyModel.query_all(ndb.Key('MyModel', '*defaultMyModel')).fetch(200)
key_list = ndb.put_multi(entities)
ndb.delete_multi(key_list)

query_all() 所做的就是查询 MyModel 并返回所有内容.

All the query_all() does is query MyModel and return everything.

我通过注释掉一些东西并运行方法做了一些测试，看起来前两行占用了预期的写入量(~200).

I've done some testing by commenting out things and running the method, and it looks like the first two lines take up the expected amount of writes (~200).

运行第三行，ndb.delete_multi()，占我每天 50,000 次写入限额的 8%，所以大约 4000 次写入——是我认为应该做的数量的 20 倍.

Running the third line, ndb.delete_multi(), takes up about 8% of my 50,000 daily write allowance, so about 4000 writes--20 times as many as I think it should be doing.

我还确保 key_list 只包含 200 个带日志记录的键.

I've also made sure the key_list contains only 200 keys with logging.

关于为什么这需要这么多写入的任何想法?我使用的方法有误吗?或者它只是使用了大量内存?在这种情况下，有什么方法可以让我更有效地做到这一点?

Any ideas on why this takes up so many writes? Am I using the method wrong? Or does it just use a ton of memory? In that case, is there any way for me to do this more efficiently?

谢谢.

推荐答案

您的代码示例效率极低.如果您要删除大量实体，则需要对以下实体进行批处理，但是，您应该使用 keys_only 查询检索数据，然后删除:

Your code example is extremely inefficient. If you are deleting large numbers of entities than you will need to batch the below but, you should be retrieving data with a keys_only query and then deleting:

from google.appengine.ext import ndb

ndb.delete_multi(
    MyModel.query().fetch(keys_only=True)
)

关于写入操作的数量(请参阅 Andrei 的回答)，确保只有模型上需要索引的字段启用了索引".

In regards to the number of write operations (see Andrei's answer), ensure only the fields on your model that are required to be indexed "have an index enabled".

这篇关于Google App Engine ndb.delete_multi() 的效率如何?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！