使用Erlang，我应该如何在集群之间分配负载？ [英] Using Erlang, how should I distribute load amongst a cluster?

问题描述

我正在看奴隶/游泳池模块，它看起来像我想要的
，但也似乎我在
应用程序中有一个单点故障（如果主节点去

客户端有一个网关列表（为了回退的目的 - 所有的
都是相同的东西），接受连接，一个被选择从
随机由客户端。当客户端连接所有节点
进行检查以查看哪个负载最小，然后将最少
加载的服务器的IP转发回客户端。客户端然后
连接到这个服务器，一切都在那里执行。

总而言之，我想让所有的节点作为两个网关，实际上是
处理客户端请求。负载平衡仅在
客户端初始连接时完成 - 所有实际数据包都在客户端的home节点的
处理。

我该怎么做？

解决方案

我不知道是否有这个模块实现了，但是我可以说，负载平衡被高估了。我可以争论的是，随机放置工作是最好的选择，除非你知道更多的信息如何负载将来将来，在大多数情况下，你真的没有。你写的是：

当客户端连接所有节点时，检查哪个节点负载最小，然后是最少加载的IP服务器被转发回客户端。

你如何知道所有这些最少加载的节点在下一个ms中将不会被加载最高？你怎么知道所有那些不包括在列表中的高加载节点不会在下一个ms中丢弃负载？你真的不知道它，除非你有非常罕见的情况。

只是衡量（或计算）你的节点的性能和设置节点的概率取决于它。随机选择节点，无论当前负载如何。使用它作为初始方法。当你设置它，那么你可以尝试弥补一些更复杂的算法。我敢打赌，打败这个初步方法将是非常困难的工作。相信我，非常努力。

编辑：为了在一个细微的细节上更加清楚，我强烈地认为你无法预测未来的负载从当前和历史负载，但您应该使用有关任务持续时间概率和任务生命周期分解的知识。这项工作很难尝试实现。

I was looking at the slave/pool modules and it seems similar to what I want, but it also seems like I have a single point of failure in my application (if the master node goes down).

The client has a list of gateways (for the sake of fallback - all do the same thing) which accept connections, and one is chosen from randomly by the client. When the client connects all nodes are examined to see which has the least load and then the IP of the least- loaded server is forwarded back to the client. The client then connects to this server and everything is executed there.

In summary, I want all nodes to act as both gateways and to actually process client requests. The load balancing is only done when the client initially connects - all of the actual packets and processed on the client's "home" node.

How would I do this?

解决方案

I don't know if there is this modules implemented yet but what I can say, load balance is overrated. What I can argue is, random placing of jobs is best bet unless you know far more information how load will come in future and in most of cases you really doesn't. What you wrote:

When the client connects all nodes are examined to see which has the least load and then the IP of the least- loaded server is forwarded back to the client.

How you know that all those least loaded node will not be highest loaded just in next ms? How you know that all those high loaded nodes which you will not include in list will not drop load just in next ms? You really can't know it unless you have very rare case.

Just measure (or compute) your node's performance and set node's probability be chosen depend of it. Choose node randomly regardless of current load. Use this as initial approach. When you set it up, then you can try make up some more sophisticated algorithm. I bet that it will be very hard work to beat this initial approach. Trust me, very hard.

Edit: To be more clear in one subtle detail, I strongly argue that you can't predict future load from current and historical load but you should use knowledge about tasks durations probability and current decomposition of task's lifetime. This work is so hard to try achieve.

这篇关于使用Erlang，我应该如何在集群之间分配负载？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！