发送失败事件EventHub [英] Failed to send event to EventHub

问题描述

我们遇到了很多这些异常流量高峰期间发送事件EventHubs的：

We are experiencing lots of these exceptions sending events to EventHubs during peak traffic:

发送失败事件EventHub例外：Microsoft.ServiceBus.Messaging.MessagingException：服务器无法处理请求，请重试操作如果问题仍然存在，请联系您的服务总线管理员，并提供跟踪。 ID。
要么
发送失败事件EventHub例外：System.TimeoutException：操作未在指定的时间内完成

"Failed to send event to EventHub. Exception : Microsoft.ServiceBus.Messaging.MessagingException: The server was unable to process the request; please retry the operation. If the problem persists, please contact your Service Bus administrator and provide the tracking id." or "Failed to send event to EventHub. Exception : System.TimeoutException: The operation did not complete within the allocated time "

您可以清楚地看到它在这里：

You can see it clearly here:

正如你所看到的，我们得到了很多内部错误，服务器忙的错误，失败请求时收到的邮件超过40万次/小时（或270〜MB /小时）。这不仅是一个瞬态问题。它明确相关的吞吐量。

As you can see, we got lots of Internal Errors, Server Busy Errors, Failed Request when Incoming messages are over 400K events/hour (or ~270 MB/hour). This is not just a transient issue. It's clearly related to throughput.

我们的EH有32个分区，7天邮件保留，并指定5吞吐量单位。 OperationTimeout设置为5分钟，而我们使用的是默认RetryPolicy。

Our EH has 32 partitions, message retention of 7 days, and 5 throughput units assigned. OperationTimeout is set to 5 mins, and we are using the default RetryPolicy.

它是什么，我们还需要在这里调整？我们真正关心的EH的可扩展性。

Is it anything we still need to tweak here? We are really concerned about the scalability of EH.

感谢

推荐答案

发送吞吐调节可以通过高效的分区分配策略来实现。没有任何单一的旋钮，它可以做到这一点。下面是基本的信息，您将需要能够以设计为高吞吐方案。

Send throughput tuning can be achieved using efficient partition distribution strategies. There isn't any single knob which can do this. Below is the basic information you will need to be able to design for High-Thruput Scenarios.

1）允许从命名空间开始：吞吐量单位（又名课时）在命名空间级别配置。 PLS。记住，那个，TU的配置被应用 - 该命名空间下的所有EventHubs的总和。如果你有5个课时你的命名空间，并在其5 eventhubs - 它会将所有5 eventhubs之间进行分配。

1) Lets start from the Namespace: Throughput Units(aka TUs) are configured at Namespace level. Pls. bear in mind, that, TUs configured is applied - aggregate of all EventHubs under that Namespace. If you have 5 TUs on your Namespace and 5 eventhubs under it - it will be divided among all 5 eventhubs.

2）现在让我们看看EventHub级别：如果EventHub分配在5课时，它有32个分区 - 没有一个单一的分区可以使用所有5课时。外汇。如果你要发送的数据5TU 1分区和零到所有其他分区31 - 这是不可能的。则需要确保该数据在所有分区分布均匀。 EventHubs支持3种类型的发送 - 这让用户在分区分配不同级别的控制：

2) Now lets look at EventHub level: If the EventHub is allocated with 5 TUs and it has 32 partitions - No single partition can use all 5 TUs. For ex. if you are trying to send 5TU of data to 1 partition and 'Zero' to all other 31 partitions - this is not possible. You will need to ensure that the data is distributed evenly across all partitions. EventHubs support 3 types of sends - which gives users different level of control on Partition distribution:

1. EventHubClient.Send（EventDataWithoutPartitionKey） - >如果您使用此API发送 - eventhub会照顾均匀分布的数据
。

2. EventHubClient.Send（EventDataWithPartitionKey） - >如果您要使用此API发送到EventHubs - 在partitionKey将决定你的数据的分布。 PartitionKey使用散列EVENTDATA相应的分区（算法中，以哈希是微软专有和不共享）。通常谁需要一组消息的相关用户将使用发送的这种变异。


3. EventHubSender.Send（EVENTDATA） - >在此版本中，发件人已经连接到分区。所以 - 这给整个分区分配给客户的完全控制权
。

要了解你的客户分布 - 你应该使用<一个href=\"https://msdn.microsoft.com/en-us/library/azure/microsoft.servicebus.messaging.eventhubclient.getpartitionruntimeinformationasync.aspx\"相对=nofollow> EventHubClient.GetPartitionRuntimeInfo API来了解哪个分区是超载。黑/白BeginSequenceNumber和LastEnqueuedSequenceNumber应该给其他人相比的分区负荷的估计值的差异。

To Understand your distribution of Client - you should use EventHubClient.GetPartitionRuntimeInfo Api to understand which Partition is overloaded. The difference b/w BeginSequenceNumber and LastEnqueuedSequenceNumber is supposed to give an estimate of that partitions load compared to others.

3）最后但并非最不重要 - 你可以调整在发送操作级别的性能（吞吐量不） - 使用的sendBatch API。
1 TU可以买到1000封邮件/秒或1MBPS的马克斯 - 你将与任何限制首先击中节流 - 这不能改变。
如果你的消息都很小 - 可以说100个字节，你只能发送1000封邮件/秒（根据TU限制） - 你先打1000次/秒的限制。然而，整体使用<一个href=\"https://msdn.microsoft.com/en-us/library/azure/microsoft.servicebus.messaging.eventhubclient.sendbatchasync.aspx\"相对=nofollow>的sendBatch API - 你可以批量让说的100byte 10封邮件，推动以同样的速度 - 1000封邮件/秒，只有100的API调用，提高了系统的终端到终端的延迟（因为它有助于服务还能够有效地坚持消息）。请记住，这里唯一的限制是最大。可以发送消息大小 - 这是256 KB（此限制将适用于你的BATCHSIZE如果您使用的sendBatch API）

3) Last but not the least - you can tune performance (not Throughput) at send operation level - using the SendBatch API. 1 TU can buy a Max of 1000 msgs/sec or 1MBPS - you will be throttled with whichever limit hits first - this cannot be changed. If your messages are small - lets say 100 bytes and you can send only 1000 msgs/sec (as per the TU limit) - you will first hit the 1000 events/sec limit. However, overall using SendBatch API - you can batch lets say 10 of 100byte msgs and push at the same rate - 1000 msgs/sec with just 100 API calls and improve the end-to-end latency of the system (as it helps service also to persist messages efficiently). Remember, the only limitation here is the Max. Msg Size that can be sent - which is 256 kb (this limit will apply on your BatchSize if you use SendBatch API).

在这种背景下，你的情况：
  - 有32个分区和5个课时 - 我真的会仔细检查分区分销策略

心连心！
SREE

HTH! Sree

这篇关于发送失败事件EventHub的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！