GAE使用后端NDB的put_multi（）实体 [英] GAE put_multi() entities using backend NDB

问题描述

我正在使用后端编写多个实体，其中 ndb.put_multi（list_of_entities）。

我遇到的问题是，如果我进行查询，那我就没有结果。如果我把例如1秒的睡眠定时器，我可以读取我刚才写的实体。

例如：

  class图片（ndb.Expando）： 
通过
 
类最爱（ndb.Expando）：
 user_id = ndb.StringProperty（必须=真）
通过
 
＃。 ..图片和收藏类型的图片列表
实体=收藏
实体[1：1] =图片
 ndb.put_multi（实体）
 
收藏=收藏。为什么在dev_appserver中返回0？query（）。filter（Favourite.user_id == user_id）.fetch（99999，keys_only = True）
 logging.info（len（favorites） 
  

首先假设问题与缓存有关。 但：

阅读多个钥匙或实体上的NDB实体操作：

高级注意事项：这些方法正确交互与上下文和
缓存;它们不直接对应特定的RPC调用。

阅读 NDB缓存
$ b

/ p>

上下文缓存仅在单个
传入HTTP请求期间持续存在，仅对处理
的代码可见请求。它很快;这个缓存存在于内存中。当NDB
函数写入数据存储区时，它还会写入上下文中的
高速缓存。当一个NDB函数读取一个实体时，它首先检查上下文中的
缓存。如果在那里找到该实体，则不会发生数据存储交互
。

查询不会在任何缓存中查找值。但是，如果高速缓存策略如此说明（但
永远不会存储到Memcache），则查询结果是
写回上下文高速缓存。

嗯，我在这里输了。一切似乎都没问题。即使查询从控制台我得到正确的总和，但从来没有在同一处理程序，无论是什么功能等。

只有我注意到的是，当等待 time.sleep（1）然后我得到正确的结果。所以这与 ndb.put_multi 可能不会同步完成的事实有关。所以很困惑....

解决方案

早晨清醒的头脑总是比晚上头脑昏昏沉沉。

谢谢大家的意见。问题解决了。您以正确的方式引导我回答我的问题：

我使用祖先查询来正确获取结果。值得一提的是

了解NDB写入：Commit，Invalidate Cache，and Apply

写数据的NDB函数（例如put（））返回缓存失效后
; 应用阶段异步发生。

这意味着在每次投入后，申请阶段可能尚未完成。

以及：

此行为会影响您的数据对您
申请。在NDB
函数返回后，这种变化可能不会完全应用到几百毫秒左右的基础数据存储区
。在应用
更改时执行的非祖先查询可能会看到不一致的状态（即部分变化但不是全部
）。有关写入时间和
查询的更多信息，请参阅App Engine中的事务隔离。

还有一些关于一致性的内容从 Google学院检索数据存储区中的数据中读取和写入之间

通过在多个数据中心同步存储数据
，Google App Engine的高复制数据存储（HRD）为您的读取和写入提供高
可用性。但是，从写入
的时间开始，直到它在所有数据中心中都可见为止，这意味着跨多个实体组（非祖先查询）的
查询只能保证
的最终结果一致。因此，
这种查询的结果有时可能无法反映最近对
基础数据的变化。 然而，通过它的关键字直接获取实体是
始终一致。

谢谢到@Paul C不断帮助，@dragonx和@sologoub帮助我理解。

I am using a backend to write multiple entities with ndb.put_multi(list_of_entities).

The issue that I am experiencing is that just after that if I make a query then I get no results. If I put a sleep timer for eg 1 sec, I can read the entities that I just wrote.

So eg:

class Picture(ndb.Expando):
    pass

class Favourite(ndb.Expando):
    user_id = ndb.StringProperty(required=True)
    pass

#...make lists with Picture and Favourite kinds
entities = favourites
entities[1:1] = pictures
ndb.put_multi(entities)

favourites = Favourite.query().filter(Favourite.user_id == user_id).fetch(99999, keys_only=True)
logging.info(len(favourites)) #returns 0 in dev_appserver why?

First assumed the problem has to do with caching. But:

Reading NDB Entities Operations on Multiple Keys or Entities:

Advanced note: These methods interact correctly with the context and caching; they don't correspond directly to specific RPC calls.

Reading NDB Caching

The In-Context Cache

The in-context cache persists only for the duration of a single incoming HTTP request and is "visible" only to the code that handles that request. It's fast; this cache lives in memory. When an NDB function writes to the Datastore, it also writes to the in-context cache. When an NDB function reads an entity, it checks the in-context cache first. If the entity is found there, no Datastore interaction takes place.

Queries do not look up values in any cache. However, query results are written back to the in-context cache if the cache policy says so (but never to Memcache).

Hm I am lost here. Everything seems to be ok. Even if query from the console I get the correct sum, but never on the same handler, no matter what function etc.

Only thing I noticed is that when put a wait time.sleep(1) then I get the correct results. So that has to do with the fact that the ndb.put_multi might not complete synchronously or not. So confused....

解决方案

A clear mind in the morning is always better than a dizzy mind at night.

Thank you all for the comments. Problem solved. You lead me in the right way so to answer my question:

I used ancestor queries to get the results correctly. It's worth to mention the following

Understanding NDB Writes: Commit, Invalidate Cache, and Apply

The NDB function that writes the data (for example, put()) returns after the cache invalidation; the Apply phase happens asynchronously.

That means that after each put the apply phase might not have completed.

And:

This behavior affects how and when data is visible to your application. The change may not be completely applied to the underlying Datastore a few hundred milliseconds or so after the NDB function returns. A non-ancestor query performed while a change is being applied may see an inconsistent state (i.e., part but not all of the change). For more information about the timing of writes and queries, see Transaction Isolation in App Engine.

Also some things about consistency between read and writes taken from Google Academy Retrieving data from the Datastore

Google App Engine's High Replication Datastore (HRD) provides high availability for your reads and writes by storing data synchronously in multiple data centers. However, the delay from the time a write is committed until it becomes visible in all data centers means that queries across multiple entity groups (non-ancestor queries) can only guarantee eventually consistent results. Consequently, the results of such queries may sometimes fail to reflect recent changes to the underlying data. However, a direct fetch of an entity by its key is always consistent.

Thanks to @Paul C for constantly helping and @dragonx and @sologoub for helping me understand.

这篇关于GAE使用后端NDB的put_multi（）实体的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！