超过2天的批量删除数据存储实体 [英] Bulk delete datastore entity older than 2 days

问题描述

我在具有字段的数据存储区中拥有实体.

I have entity in datastore with fields.

created_date = ndb.DateTimeProperty(auto_now_add = True)

created_date = ndb.DateTimeProperty(auto_now_add=True)

epoch = ndb.IntegerProperty()

epoch = ndb.IntegerProperty()

sent_requests = ndb.JsonProperty()

sent_requests = ndb.JsonProperty()

我想使用日常cron作业批量删除所有超过2天的实体.我知道ndb.delete_multi(list_of_keys)，但是如何获取早于2天的密钥列表?是否正在扫描具有100+百万个实体的整个数据存储并获取epoch < int(time.time()) - 2*86400最佳选择可用的密钥列表?

I would like bulk to delete all those entities which are older than 2 days using daily cron job. I am aware of ndb.delete_multi(list_of_keys) but how do i get list of keys which are older than 2 days? Is scanning entire datastore with 100+ million entity and getting list of keys where epoch < int(time.time()) - 2*86400 the best option available?

推荐答案

是的，因为您只想删除一些需要执行(keys_only)查询以获取要传递给ndb.delete_multi()(或其键)的键的实体异步版本?).不必担心实体的数量，所有查询都是基于索引的，响应时间并不取决于数据存储区中存在多少个实体.

Yes, because you only want to delete some of the entities you need to perform (keys_only) queries to obtain the keys to pass to ndb.delete_multi() (or its async version?). Don't worry about the number of entities, all queries are index-based, the response time doesn't depend on how many entities exist in the datastore.

但是在删除之后可能需要花费一些时间来更新索引，因此请使用查询游标，而不要重复相同的查询(这可能会返回已删除的键).

But it may take some time for the index to be updated after the deletions, so use query cursors, not repeated identical queries (which could return keys already deleted).

此外，如果您希望删除很多实体，则将负载分散到多个请求中(例如，使用任务队列或延迟库)，以防止超过请求截止时间.参见例如如何删除所有谷歌数据存储中的条目?

Also, if you expect to delete a lot of entities, spread the load in multiple requests (for example using the task queue or the deferred library) to prevent exceeding the request deadline. See, for example, How to delete all the entries from google datastore?

这篇关于超过2天的批量删除数据存储实体的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！