客观化 - 许多人在短时间内写入同一个实体，不论有无交易 [英] Objectify - many writes to same entity in short period of time with and without transaction

问题描述

我在做的是创建一个交易，其中

1）实体A的计数器更新为+1

2）A新实体B写入数据存储区。

它看起来像这样：

  WrappedBoolean result = ofy（）。transact（new Work< WrappedBoolean>（）{
 @Override 
 public WrappedBoolean run（）{
 
 // EntityA的递增计数.nu​​mEntityBs加1并保存
 entityA.numEntityBs = entityA.numEntityBs +1; 
 ofy（）。save（）。entity（entityA）.now（）; 
 
 //创建一个新的EntityB并保存它
 EntityB entityB = new EntityB（）; 
 ofy（）。save（）。entity（entityB）.now（）; 
 
 //返回一切正常
 return new WrappedBoolean（True）; 
 
} 
}）; 
  

我在做的是保持统计EntityB的entityA的数量。这两个操作需要在一个事务中，这样既可以实现保存，也不会发生。

然而，许多用户可能会执行包含以上交易。我担心我会遇到太多试图更新entityA的人的问题。这是因为如果多个事务尝试更新相同的实体，则第一个提交获胜但所有其他事务都失败。

这引出了两个问题：

<1>写了一个不好的主意，注定要在API方法上进行很多调用时才写入。有没有更好的方法来实现我想要做的事情？

2）如果有很多更新正在对一个不在事务中的实体进行（例如作为更新该实体的计数器） - 如果在很短的时间内进行大量更新，最终是否会遇到扩展问题？数据存储如何处理这个问题？

对于冗长的问题抱歉抱歉，但我希望有人能够在上述问题中阐明这个系统对我的作用。感谢。

编辑：当我的意思是很短的时间内对一个实体进行了很多更新时，请考虑一些类似Instagram的内容，以便跟踪一张图片有多少喜欢。一些用户拥有数百万的追随者，当他们发布一张新照片时，他们可以得到像10-50秒钟之类的东西。

解决方案

数据存储每个实体组允许约1个写/秒。可能不太明显的是独立实体（即没有父母和子女的实体）仍然属于一个实体组 - 它们自己。因此，对同一个独立实体的重复写入受到相同的速率限制。

超过写入限制最终会导致写入操作失败，出现类似于 TransactionFailedError（并发异常）。

重复写入同一实体在事务之外完成可以覆盖彼此。事务可以帮助解决这个问题 - 冲突的写入会自动重试几次。从这个预期来看，你的方法看起来不错。但它只有在平均写入率低于限制时才有效。

您可能想要阅读避免数据存储争用。您需要对计数器进行分片，以便能够以超过1秒的计费率对事件进行计数。

What I'm doing is creating a transaction where

1) An entity A has a counter updated to +1
2) A new entity B is written to the datastore.

It looks like this:

WrappedBoolean result = ofy().transact(new Work<WrappedBoolean>() {
    @Override
    public WrappedBoolean run() {

        // Increment count of EntityA.numEntityBs by 1 and save it
        entityA.numEntityBs = entityA.numEntityBs +1;
        ofy().save().entity(entityA).now();

        // Create a new EntityB and save it
        EntityB entityB = new EntityB();
        ofy().save().entity(entityB).now();

        // Return that everything is ok
        return new WrappedBoolean("True");

    }
});

What I am doing is keeping a count of how many EntityB's entityA has. The two operations need to be in a transaction so either both saves happen or neither happen.

However, it is possible that many users will be executing the api method that contains the above transaction. I fear that I may run into problems of too many people trying to update entityA. This is because if multiple transactions try to update the same entity, the first one to commit wins but all the others fail.

This leads me to two questions:

1) Is the transaction I wrote a bad idea and destined to cause writes not being made if a lot of calls are made to the API method? Is there a better way to achieve what I am trying to do?

2) What if there are a lot of updates being made to an entity not in a transaction (such as updating a counter the entity has) - will you eventually run into a scaling problem if a lot of updates are being made in a short period of time? How does datastore handle this?

Sorry for the long winded question but I hope someone could shed some light on how this system works for me with the above questions. Thanks.

Edit: When I mean a lot of updates being made to an entity over a short period of time, consider something like Instagram, where you want to keep track of how many "likes" a picture has. Some users have millions of followers and when they post a new picture, they can get something like 10-50 likes a second.

解决方案

The datastore allows about 1 write/second per entity group. What might not appear obvious is that standalone entities (i.e. entities with no parent and no children) still belong to one entity group - their own. Repeated writes to the same standalone entity is thus subject to the same rate limit.

Exceeding the write limit will eventually cause write ops to fail with something like TransactionFailedError(Concurrency exception.)

Repeated writes to the same entity done outside transactions can overwrite each-other. Transactions can help with this - conflicting writes would be automatically retried a few times. Your approach looks OK from this prospective. But it only works if the average write rate remains below the limit.

You probably want to read Avoiding datastore contention. You need to shard your counter to be able to count events at more than 1/second rates.

这篇关于客观化 - 许多人在短时间内写入同一个实体，不论有无交易的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！