使用后端 NDB 的 GAE put_multi() 实体 [英] GAE put_multi() entities using backend NDB
问题描述
我正在使用后端通过 ndb.put_multi(list_of_entities) 编写多个实体.
我遇到的问题是，在那之后，如果我进行查询，则不会得到任何结果.如果我将睡眠定时器设置为例如 1 秒，我就可以读取我刚刚编写的实体.
例如:
类图片(ndb.Expando):经过最喜欢的类(ndb.Expando):user_id = ndb.StringProperty(required=True)经过#...制作带有图片和收藏夹种类的列表实体 = 收藏夹实体[1:1] = 图片ndb.put_multi(实体)收藏夹 = Favourite.query().filter(Favourite.user_id == user_id).fetch(99999, keys_only=True)logging.info(len(favourites)) #为什么在 dev_appserver 中返回 0?首先假设问题与缓存有关.但是:
<块引用>高级说明:这些方法正确地与上下文和缓存；它们不直接对应于特定的 RPC 调用.
阅读NDB 缓存
<块引用>上下文缓存
上下文缓存仅在单个传入的 HTTP 请求并且仅对处理的代码可见"那个请求.它很快;这个缓存存在于内存中.当一个新开发银行函数写入数据存储区，它也写入上下文缓存.当 NDB 函数读取实体时，它会检查上下文先缓存.如果在那里找到实体，则没有 Datastore 交互发生.
查询不会在任何缓存中查找值.但是，查询结果是如果缓存策略如此写回上下文缓存(但永远不要去Memcache).
嗯，我迷路了.一切似乎都很好.即使从控制台查询我得到正确的总和，但永远不会在同一个处理程序上，无论是什么功能等.
我唯一注意到的是，当等待 time.sleep(1) 时，我会得到正确的结果.所以这与 ndb.put_multi 可能不会同步完成的事实有关.好纠结....
早上头脑清醒总比晚上头脑昏昏沉沉要好.
谢谢大家的评论.问题解决了.你以正确的方式引导我回答我的问题:
我使用祖先查询来正确获取结果.值得一提的是
<块引用>写入数据的 NDB 函数(例如 put())返回缓存失效后；应用阶段异步发生.
这意味着在每次放置之后，应用阶段可能还没有完成.
还有:
<块引用>此行为会影响数据对您可见的方式和时间应用.更改可能不会完全应用于在 NDB 之后几百毫秒左右的基础数据存储函数返回.更改时执行的非祖先查询正在应用可能会看到不一致的状态(即，部分但不是全部改变).有关写入时间的更多信息和查询，请参阅 App Engine 中的事务隔离.
还有一些关于从 Google Academy 从数据存储区检索数据的读取和写入之间的一致性的事情/p><块引用>
Google App Engine 的高复制数据存储 (HRD) 提供高通过同步存储数据为您的读写提供可用性在多个数据中心.但是，从写入时间开始的延迟是提交直到它在所有数据中心都可见意味着跨多个实体组的查询(非祖先查询)只能保证最终一致的结果.因此，结果此类查询有时可能无法反映对基础数据.但是，通过实体的键直接获取实体是始终一致.
感谢@Paul C 的不断帮助以及@dragonx 和@sologoub 帮助我理解.
I am using a backend to write multiple entities with ndb.put_multi(list_of_entities).
The issue that I am experiencing is that just after that if I make a query then I get no results. If I put a sleep timer for eg 1 sec, I can read the entities that I just wrote.
So eg:
class Picture(ndb.Expando):
pass
class Favourite(ndb.Expando):
user_id = ndb.StringProperty(required=True)
pass
#...make lists with Picture and Favourite kinds
entities = favourites
entities[1:1] = pictures
ndb.put_multi(entities)
favourites = Favourite.query().filter(Favourite.user_id == user_id).fetch(99999, keys_only=True)
logging.info(len(favourites)) #returns 0 in dev_appserver why?
First assumed the problem has to do with caching. But:
Reading NDB Entities Operations on Multiple Keys or Entities:
Advanced note: These methods interact correctly with the context and caching; they don't correspond directly to specific RPC calls.
Reading NDB Caching
The In-Context Cache
The in-context cache persists only for the duration of a single incoming HTTP request and is "visible" only to the code that handles that request. It's fast; this cache lives in memory. When an NDB function writes to the Datastore, it also writes to the in-context cache. When an NDB function reads an entity, it checks the in-context cache first. If the entity is found there, no Datastore interaction takes place.
Queries do not look up values in any cache. However, query results are written back to the in-context cache if the cache policy says so (but never to Memcache).
Hm I am lost here. Everything seems to be ok. Even if query from the console I get the correct sum, but never on the same handler, no matter what function etc.
Only thing I noticed is that when put a wait time.sleep(1) then I get the correct results. So that has to do with the fact that the ndb.put_multi might not complete synchronously or not. So confused....
A clear mind in the morning is always better than a dizzy mind at night.
Thank you all for the comments. Problem solved. You lead me in the right way so to answer my question:
I used ancestor queries to get the results correctly. It's worth to mention the following
Understanding NDB Writes: Commit, Invalidate Cache, and Apply
The NDB function that writes the data (for example, put()) returns after the cache invalidation; the Apply phase happens asynchronously.
That means that after each put the apply phase might not have completed.
And:
This behavior affects how and when data is visible to your application. The change may not be completely applied to the underlying Datastore a few hundred milliseconds or so after the NDB function returns. A non-ancestor query performed while a change is being applied may see an inconsistent state (i.e., part but not all of the change). For more information about the timing of writes and queries, see Transaction Isolation in App Engine.
Also some things about consistency between read and writes taken from Google Academy Retrieving data from the Datastore
Google App Engine's High Replication Datastore (HRD) provides high availability for your reads and writes by storing data synchronously in multiple data centers. However, the delay from the time a write is committed until it becomes visible in all data centers means that queries across multiple entity groups (non-ancestor queries) can only guarantee eventually consistent results. Consequently, the results of such queries may sometimes fail to reflect recent changes to the underlying data. However, a direct fetch of an entity by its key is always consistent.
Thanks to @Paul C for constantly helping and @dragonx and @sologoub for helping me understand.
这篇关于使用后端 NDB 的 GAE put_multi() 实体的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
