根据集群负载调整消息频率 [英] Tuning message frequency based on cluster load

问题描述

我正在使用Akka 2.2.3和Scala编写一个简单的同质集群应用程序；一种粒子过滤算法，其中每个节点在随机的时间点与其他群集成员共享数据。目前，它是一个研究应用程序，而不是关键业务系统。

I am writing a simple homogenous cluster application using Akka 2.2.3 and Scala; a particle filtering algorithm in which each node shares data with other cluster members at random points in time. It's currently a research application, not a business critical system.

目前，每个节点每秒都会向随机选择的节点发送固定大小的消息。可行，但我担心扩展时的性能（例如，云与本地）

At present, every node sends a fixed size message to a randomly selected node each second. This works, but I have concerns regarding performance when scaling (e.g. cloud versus local)

• 节点发送数据可能会过载


• 节点可能会过载其他群集成员的传入消息


• 网络可能成为瓶颈

我想在不同的网络上运行具有大型集群的应用程序，并获得良好的性能，而无需手动调整/监视。 我可以采用哪些简单的方法来调整消息的大小和频率以减轻上述担忧？

I'd like to run the application with sized clusters on different networks and and achieve good performance without manual tuning/monitoring. What simple approaches could I take to tuning message size and frequency to mitigate the above concerns?

推荐答案

您可以尝试使用自适应负载平衡 Akka群集感知路由器。

You can try to use Adaptive Load Balancing of Akka cluster-aware routers.

或者您可以尝试创建自己的机制来减少节点过载。例如，它可能包含您的Receiver actor的自定义邮箱，该邮箱可能会定期向邮箱队列的开头添加一条包含当前邮箱队列长度的消息。然后，您的Receiver actor可以将此类消息中继到所有群集节点。在这种情况下，您将能够维护Receiver邮箱使用情况的整个群集范围的统计信息（只需记住在该统计信息中包括TTL，并且不将收到的统计信息视为空邮箱指示符即可。）

Or you can try to create your own mechanism for reduce nodes overload. For example it may contain a custom mailbox for your Receiver actor which may periodically add a message containing the current mailbox queue length to the head of the mailbox queue. Then your Receiver actor may relay such a message to all cluster nodes. In this case you will be able to maintain cluster-wide statistics of Receiver's mailboxes usage (just remember to include TTL in that statistics and treat no statistics message received as empty mailbox indicator).

如果您的应用程序可以忍受部分数据丢失，那么您可以直接从接收者的邮箱中直接丢弃多余的消息（同样，这将需要自定义邮箱实现）。

If your application can tolerate partial data losses then you can simply drop extra messages directly from the Receiver's mailboxes (as well will require a custom mailbox implementation).

这篇关于根据集群负载调整消息频率的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！