Ndb强大的一致性和频繁的写入 [英] Ndb strong consistency and frequent writes

问题描述

我试图用python实现与ndb的强大一致性。
看起来我错过了一些东西，因为我的阅读表现得像他们不一致。

查询是：

  links = Link.query（ancestor = lead_key）.filter（Link.last_status == 
 None）.fetch（keys_only = True）
 
如果链接：
 do_action（）
  
 
 $ b 
关键结构是：
 
 pre $ code>领导根（通用键） - >铅 - >网站（每个领先一个） - >链接
  

我有很多使用TaskQueue并发执行的任务，并且在每个任务。有时我在更新 last_status 字段时收到太多争用异常，但我使用重试处理它。它可以打破强大的一致性吗？

预期的行为是在没有链接时调用 do_action（） last_status 等于无。实际行为是不一致的：有时 do_action（）被调用两次，有时根本不会调用。 方案

使用祖先密钥获得强一致性有一个局限性：每个实体组每秒更新一次。解决此问题的一种方法是分割实体组。 分片计数器描述了这种技术。这是一篇旧文章，但据我所知，建议仍然健全。

I'm trying to achieve strong consistency with ndb using python. And looks like I'm missing something as my reads behave like they're not strongly consistent.

The query is:

links = Link.query(ancestor=lead_key).filter(Link.last_status == 
None).fetch(keys_only=True)

if links: 
    do_action() 

The key structure is:

Lead root (generic key) -> Lead -> Website (one per lead) -> Link

I have many tasks that are executed concurrently using TaskQueue and this query is performed at the end of every task. Sometimes I'm getting "too much contention" exception when updating the last_status field but I deal with it using retries. Can it break strong consistency?

The expected behavior is having do_action() called when there are no links left with last_status equal to None. The actual behavior is inconsistent: sometimes do_action() is called twice and sometimes not called at all.

解决方案

Using an ancestor key to get strong consistency has a limitation: you're limited to one update per second per entity group. One way to work around this is to shard the entity groups. Sharding Counters describes the technique. It's an old article, but as far as I know, the advise is still sound.

这篇关于Ndb强大的一致性和频繁的写入的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！