高效的Google PubSub发布 [英] Efficient Google PubSub Publishing
问题描述
PubSub的文档指出解码后的最大有效负载为10MB.我的问题是在发布之前压缩发布者的有效负载以增加数据吞吐量是否有利?
The docs for PubSub state that the max payload after decoding is 10MB. My question is whether or not it is advantageous to compress the payload at the publisher before publishing to increase data throughput?
如果有效负载具有较高的压缩率(例如json格式的有效负载)，则这尤其有用.
This especially can be helpful if the payload has a high compression ratio like a json formatted payload.
推荐答案
如果您要在PubSub上寻求效率，我首先会专注于使用最佳的API，那就是gRPC.如果使用的是客户端库，那么无论如何都使用gRPC的机会很高.为什么要使用gRPC?
If you are looking for efficiency on PubSub I would first concentrate on using the best API, and that's the gRPC one. If are using the client libraries then the chance is high that it's using gRPC anyway. Why gRPC?
- gRPC是二进制的，您的有效负载无需经过箍即可被浸入
- REST需要将有效负载设置为base64，从而使其变得更大并具有额外的编码步骤
第二，如果可能，我将尝试对消息进行批处理，以减少呼叫数量，消除一些延迟.
Second I would try to batch the message if possible, making the number of calls lower, eliminating some latency.
最后，我将介绍压缩，但这意味着您需要在订户处专门对其进行解压缩.这意味着您的应用程序代码将变得更加复杂.如果您所有的工作负载都在Google Cloud Platform上，那么我就不会为压缩而烦恼.如果您的工作负载不在GCP范围内，则可以考虑使用它，但是进行测试很有意义.
And last I would look at compression, but that means you need to specifically de-compress it at the subscriber. This means your application code will get more complex. If all your workloads are on the Google Cloud Platform I wouldn't bother with compression. If your workload is outside of GCP you might consider it, but testing would make sense.
一种压缩方法，如果您的模式是稳定的，那么可以考虑使用ProtoBuf.
An alternative for compression and if your schema is stable, is looking at using ProtoBuf.
最后，我会:
- 确保您使用的是gRPC
- 可能的话
- 仅在需要时和基准测试之后压缩(这意味着应用程序中有额外的逻辑)
这篇关于高效的Google PubSub发布的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
