Google PubSub同时发布请求 [英] Google PubSub Simultaneous Publish Requests

问题描述

在Google PubSub中，可以异步调用来自客户端的发布调用.因此，我认为有可能同时触发多个发送请求并将其发送到服务器，尤其是在批处理阈值太低的情况下.

In Google PubSub, the publish call from the client can be called asynchronously. Because of this, I would think that it would be possible to have multiple publish requests triggered and sent to the server, all at the same time, especially if the batch thresholds are too low.

如果这是真的，那么pubsub客户端如何控制可以创建的同时发布请求的数量?是否存在硬限制，或者如果创建太多请求，可能会发生错误?这是拥有异步发布者的预期用途，还是仅仅是为了允许发生其他非发布活动?

If this is true, how does the pubsub client control the number of simultaneous publish requests that can be created? Is there a hard limit, or an error that can occur if too many requests are created? Is this the intended use of having an asynchronous publisher, or is simply to allow for other non-publishing activity to occur?

尽管此问题适用于所有客户端，但我们特别遇到C#客户端问题，并间歇性地收到以下错误:

Though this question applies to any of the clients, we are specifically having an issue with the C# client, and are intermittently receiving the following error:

 Grpc.Core.RpcException: Status(StatusCode=DeadlineExceeded, Detail="Deadline Exceeded")
 at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
 at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
 at Google.Api.Gax.Grpc.ApiCallRetryExtensions.<>c__DisplayClass0_0`2.<<WithRetry>b__0>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---

我的想法是我们发送了太多的发布请求...，但我不确定.

My thought is that we are sending too many publish requests..., but I am not sure.

推荐答案

我建议使用原始gRPC代码，但要使用具有非常薄包装的客户端库.

I would advice using the raw gRPC code, but use the client library that has a very thin wrapper.

查看客户端源代码总是对我有帮助，您可以在此处找到c#代码

Looking at the client source code always helps me, you can find for c# code here PublisherClient.cs (thin wrapper)

如果您正在使用PublishAsync，它无论如何都会对消息进行排队/批处理，则该行为将控制您提供给客户端的设置(有关如何调整它的信息，请参见PublisherServiceApiClient).您还可以控制用于在客户端中发送队列的客户端连接数.我建议先使用批处理大小，然后再使用连接数，直到找到吞吐量的最佳位置为止.

If you are using PublishAsync it queues/batch the messages anyway, the behaviour is controlled the settings you give to the client (see PublisherServiceApiClient for how to tune it). You can also control the number of client connections that are used to send the queues in the client. I suggest playing with the batch-size first, then the number of connections till you found your sweet spot for your throughput.

这篇关于Google PubSub同时发布请求的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！