您能以多快的速度自动增加Firebase实时数据库上的值? [英] How quickly can you atomically increment a value on the Firebase Realtime Database?

问题描述

firebaser here

最近我发了推文关于Firebase实时数据库中新的increment()运算符时，队友问increment()有多快.

When I recently tweeted about the new increment() operator in the Firebase Realtime Database, a team mate asked how fast increment() is.

我一直在想同样的事情:用increment(1)递增值有多快?与使用事务来增加值相比， a>?

I've been wondering the same: how fast can you increment a value with increment(1)? And how does that compare to using a transaction to increment a value?

推荐答案

TL; DR

我测试了这些情况:

TL;DR

I tested these cases:

1. 通过transaction调用增加值:

ref.transaction(function(value) {
  return (value || 0) + 1;
});

使用新的increment运算符递增值:

Increment a value with the new increment operator:

ref.set(admin.database.ServerValue.increment(1));

增加更快的事实不足为奇，但是...增加了多少?

The fact that increment is faster won't be a surprise, but... by how much?

结果:

• 通过事务，我每秒可以增加大约60-70次的值.
• 使用increment运算符，我每秒可以将值增加200-300次.

• With transactions I was able to increment a value about 60-70 times per second.
• With the increment operator, I was able to increment a value about 200-300 times per second.

我已经在我的2016年型号MacBook Pro上进行了测试，并将以上内容包装在使用

I've run the test on my 2016 model macBook pro, and wrapping the above in a simple Node.js script that uses the client-side Node SDK. The wrapping script for the operations was really basic as well:

timer = setInterval(function() {
    ... the increment or transaction from above ...
}, 100);

setTimeout(function() {
  clearInterval(timer);
  process.exit(1);
}, 60000)

因此:每秒将值增加10次，并在1分钟后停止执行.然后，我使用以下脚本生成此过程的实例:

So: increment the value 10 times per second, and stop doing that after 1 minute. I then spawned instances of this process with this script:

for instance in {1..10}
do
  node increment.js &
done

因此，这将使用increment运算符运行10个并行进程，每个进程每秒将值增加10次，总计每秒增加100次.然后，我更改了实例数，直到每秒增量"达到峰值.

So this would run 10 parallel processes with the increment operator, each increasing the value 10 times per second, for a total of 100 increments per second. I then changed the number of instances until the "increments per second" reached a peak.

然后，我在jsbin上写了一个小的脚本来监听值，并通过一个简单的低通移动平均滤波器确定每秒的增量数.我在这里遇到了麻烦，因此不确定计算是否完全正确.鉴于我的测试结果，他们已经足够接近了，但是如果有人想写一个更好的观察者:请成为我的客人. :)

I then wrote a small script on jsbin to listen for the value, and determine the number of increments per second by a simple low pass, moving average filter. I had some trouble here, so am not sure if the calculations are completely correct. Given my test results they were close close enough, but if anyone feels like writing a better observer: be my guest. :)

有关测试的注意事项:

1. 我一直在增加进程数量，直到每秒增量"似乎达到最大值，但我注意到这与我的笔记本电脑风扇全速运行的时间相吻合.因此，我可能没有找到服务器端操作的真正最大吞吐量，而是找到了我的测试环境和服务器的结合.因此，当您尝试重现此测试时，很有可能(实际上很可能)会得到不同的结果，尽管increment的吞吐量当然应该总是明显高于transaction.不管您得到什么结果:请分享. :)

1. I kept increasing the number of processes, until the "increments per second" seemed to max out, but I noticed that this coincided with my laptop fans going full-speed. So it's likely that I didn't find the true maximum throughput of the server-side operation, but a combination of my test environment and the server. So it is quite possible (and in fact likely) you may get different results when you try to reproduce this test, although of course the increment throughput should always be significantly higher than the transaction. No matter what results you get: please share them. :)

我已经使用了客户端Node.js SDK，因为它最容易上手.使用不同的SDK可能会产生稍微不同的结果，尽管我希望主要的SDK(iOS，Android和Web)与我所得到的非常接近.

I've used the client-side Node.js SDK, as it was easiest to get working. Using different SDKs may give slightly different results, although I expect the primary SDKs (iOS, Android, and Web) to be quite close to what I got.

两个不同的队友立即问我是否要在单个节点上运行它，或者我是否正在并行增加多个值.并行增加多个值可能会显示是否存在系统范围的吞吐量瓶颈，或者它是否是特定于节点的(我希望如此).

Two different team mates immediately asked whether I'd run this on a single node, or if I was incrementing multiple values in parallel. Incrementing multiple values in parallel might show if there's a system-wide throughput bottleneck in or if it is node-specific (which I expect).

如前所述:我的测试工具没什么特别的，但是我的jsbin观察者代码尤其令人怀疑.如果有人想在相同的数据上编写更好的观察者，则表示敬意.

As said already: my test harness is nothing special, but my jsbin observer code is especially suspect. Kudos if anyone feels like coding up a better observer on the same data.

---

交易和增量操作员如何在后台工作

要了解transaction和increment之间的性能差异，确实有助于了解这些操作的工作原理.对于Firebase Realtime Database，在幕后"意味着通过Web Socket连接在客户端和服务器之间发送的命令和响应.

---

How the transaction and increment operator work under the hood

To understand the performance difference between transaction and increment it really helps to know how these operations work under the hood. For the Firebase Realtime Database "under the hood" means, the commands and responses that are sent between the clients and server over the Web Socket connection.

交易使用比较并设置方法.每当我们像上面那样开始事务时，客户端都会猜测节点的当前值.如果从未看到该节点，则该猜测为null.它用这个猜测调用事务处理程序，然后我们的代码返回新值.客户端将猜测和新值发送到服务器，服务器执行比较并设置操作:如果猜测正确，请设置新值.如果猜错了，服务器将拒绝该操作，并将实际的当前值返回给客户端.

Transactions in Firebase use a compare-and-set approach. Whenever we start transaction like above, the client takes a guess at the current value of the node. If it's never see the node before that guess is null. It calls our transaction handler with that guess, and our code then returns the new value. The client send the guess and the new value to the server, which performs a compare-and-set operation: if the guess is correct, set the new value. If the guess is wrong, the server rejects the operation and returns the actual current value to the client.

在理想情况下，最初的猜测是正确的，并且该值会立即写入服务器上的磁盘(然后，将其发送给所有侦听器).在这样的流程图中:

In a perfect scenario, the initial guess is correct, and the value is immediately written to disk on the server (and after that, sent out to all listeners). In a flow chart that'd look like this:

            Client            Server

               +                   +
 transaction() |                   |
               |                   |
        null   |                   |
     +---<-----+                   |
     |         |                   |
     +--->-----+                   |
         1     |     (null, 1)     |
               +--------->---------+
               |                   |
               +---------<---------+
               |     (ack, 3)      |
               |                   |
               v                   v

但是，如果节点在服务器上已经有一个值，它将拒绝写入，发送回实际值，然后客户端再次尝试:

But if the node already has a value on the server, it rejects the write, sends back the actual value, and the client tries again:

            Client            Server

               +                   +
 transaction() |                   |
               |                   |
        null   |                   |
     +---<-----+                   |
     |         |                   |
     +--->-----+                   |
         1     |                   |
               |     (null, 1)     |
               +--------->---------+
               |                   |
               +---------<---------+
               |     (nack, 2)     |
               |                   |
         2     |                   |
     +---<-----+                   |
     |         |                   |
     +--->-----+                   |
         3     |      (2, 3)       |
               +--------->---------+
               |                   |
               +---------<---------+
               |      (ack, 3)     |
               |                   |
               |                   |
               v                   v

这还算不错，一个额外的往返.即使Firebase会使用悲观锁定，也需要该往返，所以我们不会丢失任何东西.

This isn't too bad, one extra roundtrip. Even if Firebase would've used pessimistic locking, it would have needed that roundtrip, so we didn't lose anything.

如果多个客户端同时修改同一值，则问题开始.这会在节点上引入所谓的竞争，如下所示:

The problem starts if multiple clients are modifying the same value concurrently. This introduces so-called contention on the node, which looks like this:

            Client            Server                Client
               +                   +                   +
 transaction() |                   |                   |
               |                   |                   | transaction()
        null   |                   |                   |
     +---<-----+                   |                   |  null
     |         |                   |                   +--->----+
     +--->-----+                   |                   |        |
         1     |                   |                   +---<----+ 
               |     (null, 1)     |                   |   1
               +--------->---------+    (null, 1)      |
               |                   |---------<---------+
               +---------<---------+                   |
               |     (nack, 2)     |--------->---------+
               |                   |     (nack, 2)     |
         2     |                   |                   |
     +---<-----+                   |                   |   2
     |         |                   |                   |--->----+
     +--->-----+                   |                   |        |
         3     |      (2, 3)       |                   |---<----+ 
               +--------->---------+                   |   3
               |                   |                   |
               +---------<---------+                   |
               |      (ack, 3)     |       (2, 3)      |
               |                   |---------<---------+
               |                   |                   |
               |                   |--------->---------+
               |                   |    (nack, 3)      |
               |                   |                   |   3
               |                   |                   |--->----+
               |                   |                   |        |
               |                   |                   |---<----+ 
               |                   |                   |   4
               |                   |       (3, 4)      |
               |                   |---------<---------+
               |                   |                   |
               |                   |--------->---------+
               |                   |     (ack, 4)      |
               |                   |                   |
               v                   v                   v

待办事项:更新以上图表，以使服务器上的操作不会重叠.

第二个客户端必须对其操作进行另一次重试，因为服务器端的值已在其第一次尝试和第二次尝试之间进行了修改.我们写给该位置的客户越多，您看到重试的可能性就越大. Firebase客户端会自动执行这些重试，但是在重试多次之后，它将放弃并向应用程序引发Error: maxretry异常.

The second client had to do another retry for its operation, because the server-side value had been modified between its first and second try. The more clients we have writing to this location, the more likely it is that you'll see retries. And the Firebase client performs those retries automatically, but after a number of retries it will give up and raise an Error: maxretry exception to the application.

这就是为什么我每秒只能增加一个计数器约60-70次的原因:写操作多于此，节点上的争用就太多了.

This is the reason I could only increment a counter about 60-70 times per second: with more writes than that, there was too much contention on the node.

递增操作本质上是原子的.您正在告诉数据库:无论当前值是什么，都将其提高x.这意味着客户端永远不必知道节点的当前值，因此也不会猜错.它只是告诉服务器该怎么做.

An increment operation is atomic by nature. You're telling the database: whatever the current value is, make it x higher. This means that the client never has to know the current value of the node, and so it also can't guess wrong. It simply tells the server what to do.

使用increment时，我们具有多个客户端的流程图如下:

Our flow chart with multiple clients looks like this when using increment:

            Client            Server                Client

               +                   +                   +
  increment(1) |                   |                   |
               |                   |                   | increment(1)
               |  (increment, 1)   |                   |
               +--------->---------+   (increment, 1)  |
               |                   |---------<---------+
               +---------<---------+                   |
               |      (ack, 2)     |--------->---------+
               |                   |     (ack, 3)      |
               |                   |                   |
               v                   v                   v

仅这最后两个流程图的长度就已经很长了，可以解释为什么increment在这种情况下如此快:increment操作是为此目的而完成的，因此有线协议更紧密地表示了什么我们正在努力实现.而这种简单性仅在我的简单测试中就导致3到4倍的性能差异，在生产场景中甚至可能更多.

The length of these last two flow charts alone already goes a long way to explain why increment is so much faster in this scenario: the increment operation is made for this, so the wire protocol much more closely represents what we're trying to accomplish. And that simplicity leads to a 3x-4x performance difference in my simple test alone, and probably even more in production scenarios.

当然，事务仍然有用，因为原子操作远不止是增加/减少.

Of course transactions are still useful, as there are many more atomic operations than just increments/decrements.

这篇关于您能以多快的速度自动增加Firebase实时数据库上的值?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！