Google Club Pub / Sub比较对象更改通知的可扩展性如何 [英] How scalable are Google Club Pub/Sub compared Object Change Notifications

问题描述

正如标题所问，使用签名URL来上传对象时，Google Club Pub / Sub与Google云端存储中的对象更改通知相比的可扩展性如何？ 在能够处理很多在短时间内上传的对象方面，每个对比都是如此？如果很多对象都很快上传，交付速度会变慢吗？例如，1000个对象/秒？

如果这两个选项都不可扩展，那么还有其他选项？为了我的目的，我需要上传图片，那么当通知发送到我的Google App Engine应用程序时，我需要写入我的数据库。上传图像和通知之间的时间间隔很短（最多2-3秒）是非常重要的。

解决方案

p>在每秒1000次对象更改时，您希望使用Cloud Pub / Sub通知。

对象更改通知和云端发布/ QPS。但是，在这种负载下，有几个理由更喜欢Cloud Pub / Sub。

首先，Cloud Pub / Sub订阅支持拉取消息。通过一次调用pull（），您可以一次检索100条或更多条消息，然后通过一次调用acknowledge（）来确认它们。对象更改通知通常会为每封邮件调用一次服务。使用Cloud Pub / Sub可以立即将服务器需要处理的RPC数量减少两个数量级。其次，在QPS高的情况下，您需要启动考虑失败，超时和重试。由于各种原因，Cloud Pub / Sub也是一个优越的选择。首先，它支持可配置的确认最后期限，而对象更改通知必须在20秒内处理。另一方面，您可以查询查看当前积压的大小，以防万一您落后。如果您使用推送订阅，如果您的消息接收器脱机一段时间，Cloud Pub / Sub也会有更加友好的流量增长，所以当您重新联机时，您不会压倒自己的服务器。

三，灵活性。 Cloud Pub / Sub是围绕这个用例而设计的。接收大量的消息是他们的全部面包和黄油，并且有许多关注它的功能和库。但云存储专注于存储数据。对象更改通知有效，但绝不会提供与Cloud Pub / Sub一样多的通知功能。

As the title asks, how scalable is Google Club Pub/Sub compared to Object Change Notifications in Google Cloud Storage when using Signed URLs to upload objects?

How do each compare in terms of being able to handle many objects being uploaded in a short period of time? Will delivery be slower if many objects are uploaded quickly? For example, 1000 objects/second?

If neither are scalable what other options are there?

For my purposes, I need to upload an image, then when a notification is delivered to my Google App Engine app, I need to make a write to my database. It is essential that the period of time between uploading the image and the notification is short (2-3 seconds at the most).

解决方案

At 1000 object changes per second, you want to use Cloud Pub/Sub notifications.

Both Object Change Notifications and Cloud Pub/Sub notifications will work just fine at 1000 QPS. However, at those sorts of load, there are a few reasons to prefer Cloud Pub/Sub.

First, Cloud Pub/Sub subscriptions support pulling messages. With one call to pull(), you can retrieve 100 or more messages at once, and then acknowledge them all with one call to acknowledge(). Object Change Notifications always make one call to your service per message. Using Cloud Pub/Sub can instantly reduce the number of RPCs your server needs to deal with by two orders of magnitude.

Second, at high QPS, you will want to start considering failures, timeouts, and retries. Cloud Pub/Sub is a superior option here as well, for a variety of reasons. For one, it supports configurable ack deadlines, while object change notifications always must be processed within 20 seconds. For another, you can query to see the size of your current backlog, in case you get behind. If you're using push subscriptions, Cloud Pub/Sub also has a much friendlier traffic ramp-up if your message receiver goes offline for a bit, so you don't overwhelm your own servers as they come back online.

Third, flexibility. Cloud Pub/Sub is designed around this use case. Receiving a high volume of messages is their whole bread and butter, and there are many features and libraries focused on it. Cloud Storage, though, is focused on storing data. Object change notifications works, but it will never offer as many notification features as Cloud Pub/Sub.

这篇关于Google Club Pub / Sub比较对象更改通知的可扩展性如何的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！