Erlang节点到节点消息吞吐量，超时和保证 [英] Erlang Node to Node Messaging Throughput, Timeouts and guarantees

问题描述

现在，假设我们正在设计一个应用程序，由2个Erlang节点组成。在节点A上，将有很多进程，数千个。这些进程通过向节点B上的已注册进程发送消息来访问节点B上的资源。

在节点B中，假设您通过执行以下功能开始进程：

 
 start_server（） - > 
注册（zeemq_server，spawn（？MODULE，server，[]）），好的。
 
 server（） - > 
接收
 {{CallerPid，Ref}，{Module，Func，Args}}  - > 
 Result =（catch erlang：apply（Module，Func，Args）），
 CallerPid！ {Ref，Result}，
 server（）; 
 _  - > server（）
结束。 
 

在节点A上，任何想要在节点B上给定模块中执行任何函数的进程使用以下代码：

 
调用（Node，Module，Func，Args） - > 
 Ref = make_ref（），
 Me = self（），
 {zeemq_server，Node}！ {{Me，Ref}，{Module，Func，Args}}，
收到
 {Ref，Result}  - >结果
 after timer：minutes（3） - > 
 error_logger：error_report（[Call to server take long long]），
 {error，remote_call_failed} 
 end。 
 

因此，假设Node B上的Process zeemq_server 将永远不会关闭，并且节点A和B一直在上，请回答以下问题：


Qn 1 ：由于节点B上只有一个接收进程，因此其邮箱最有可能一直都满了这是因为，节点A上的进程很多，在给定的时间间隔（例如2秒），至少每个进程都对Node B服务器进行一次调用。在节点B上哪个方式可以使接收冗余？ ，例如过程组e.t.c.并解释（这个概念）这将如何取代上面的服务器端代码。显示客户端会发生什么变化。


Qn 2 ：在节点B上只有一个接收器的情况下，是否有最大数量的消息允许在进程邮箱？如果一个流程邮件淹没了太多的邮件，那么erlang会如何回应？


Qn 3 ：使用上述概念，我可以保证发送请求的每个进程尽快回复答案在超时发生之前？可以将节点B上的接收部分转换为并行操作帮助？像这样：

 
 start_server（） - > 
注册（zeemq_server，spawn（？MODULE，server，[]）），好的。
 
 server（） - > 
接收
 {{CallerPid，Ref}，{Module，Func，Args}}  - > 
  spawn（？MODULE，child，[Ref，CallerPid，{Module，Func，Args}]）， 
 server（）; 
 _  - > server（）
 end。
 
 child（Ref，CallerPid，{Module，Func，Args}） - > 
 Result =（catch erlang：apply（Module，Func，Args）），
 CallerPid！ {Ref，Result}，
 ok。 
 

上面显示的方法可能会增加节点B上运行的进程的即时次数，这可能会因内存而大大影响服务。但是，它看起来不错，并使 server（）循环立即返回以处理下一个请求。你对这个修改有什么看法？


最后：说明如何在节点B上实现接收者线程的池，但似乎是在节点A之下的一个名称之下。这样，传入消息在接收器线程和在该组进程之间共享的负载之间被复用。保持问题的意义相同。

谢谢 Erlangers ！

解决方案

除了内存量之外，进程邮箱中的最大邮件数是无限制的。

另外，如果您需要检查邮箱大小，使用

  erlang：process_info（self（），[message_queue_len，messages]）。 
  

这将返回以下内容：

  [{message_queue_len，0}，{messages，[]}] 
  

我建议你先把你的服务器转换成gen_server。这是你的工作人员。

接下来，我建议使用poolboy（ https ：//github.com/devinus/poolboy ）创建一个作为poolboy工作器的服务器实例池（在他们的github Readme.md中有例子）。最后，我建议使用帮助方法创建一个模块，创建一个poolboy事务，并将一个Worker arg从池中应用到一个函数。下面的例子来自他们的github：

  squery（PoolName，Sql） - > 
 poolboy：transaction（PoolName，fun（Worker） - > 
 gen_server：call（Worker，{squery，Sql}）
 end） 
  

那么说，Erlang RPC能否更好地满足您的需求？有关Erlang RPC的详细信息，请访问 http://www.erlang.org/doc/man/rpc。 HTML 。 Erlang RPC的一个很好的治疗方法可以在 http://learnyousomeerlang.com/distribunomicon#rpc 找到。

Now, suppose we are designing an application, consists of 2 Erlang Nodes. On Node A, will be very many processes, in the orders of thousands. These processes access resources on Node B by sending a message to a registered process on Node B.

At Node B, lets say you have a process started by executing the following function:

start_server()->
    register(zeemq_server,spawn(?MODULE,server,[])),ok.

server()->
    receive
            {{CallerPid, Ref}, {Module, Func, Args}} ->
                    Result = (catch erlang:apply(Module, Func, Args)),
                    CallerPid ! {Ref, Result},
                    server();
            _ -> server()
    end.

On Node A, any process that wants to execute any function in a given module on Node B, uses the following piece of code:

call(Node, Module, Func, Args)->
        Ref = make_ref(),
        Me = self(),
        {zeemq_server,Node} ! {{Me, Ref}, {Module, Func, Args}},
        receive
                {Ref, Result} -> Result
        after timer:minutes(3) -> 
            error_logger:error_report(["Call to server took so long"]),
            {error,remote_call_failed}
        end.

So assuming that Process zeemq_server on Node B, will never be down, and that the network connection between Node A and B is always up, please answer the following questions:

Qn 1: Since there is only one receiving process on Node B, its mail box is most likely to be full , all the time. This is because, the processes are many on Node A and at a given interval, say, 2 seconds, every process at least ,makes a single call to the Node B server. In which ways, can the reception be made redundant on the Node B ? , e.g. Process Groups e.t.c. and explain (the concept) how this would replace the server side code above. Show what changes would happen on the Client side.

Qn 2: In a situation where there is only one receiver on Node B, is there a maximum number of messages allowable in the process mail box ? how would erlang respond , if a single process mail ox is flooded with too many messages ?

Qn 3: In what ways, using the very concept showed above, can i guarantee that every process which sends a request , gets back an answer as soon as possible before the timeout occurs ? Could converting the reception part on the Node B to a parallel operation help ? like this:

start_server()->
    register(zeemq_server,spawn(?MODULE,server,[])),ok.

server()->
    receive
            {{CallerPid, Ref}, {Module, Func, Args}} ->
                    spawn(?MODULE,child,[Ref,CallerPid,{Module, Func, Args}]),
                    server();
            _ -> server()
    end.

child(Ref,CallerPid,{Module, Func, Args})->
    Result = (catch erlang:apply(Module, Func, Args)),
    CallerPid ! {Ref, Result},
    ok.

The method showed above, may increase the instantaneous number of processes running on the Node B, and this may affect the service greatly due to memory. However, it looks good and makes the server() loop to return immediately to handle the next request. What is your take on this modification ?

Lastly : Illustrate how you would implement a Pool of receiver Threads on Node B, yet appearing to be under one Name as regards Node A. Such that, incoming messages are multiplexed amongst the receiver threads and the load shared within this group of processes. Keep the meaning of the problem the same.

Thank you Erlangers !

解决方案

The maximum number of messages in a process mailbox is unbounded, except by the amount of memory.

Also, if you need to inspect the mailbox size, use

erlang:process_info(self(),[message_queue_len,messages]).

This will return something like:

[{message_queue_len,0},{messages,[]}]

What I suggest is that you first convert your server above into a gen_server. This your worker.

Next, I suggest using poolboy ( https://github.com/devinus/poolboy ) to create a pool of instances of your server as poolboy workers (there are examples in their github Readme.md). Lastly, I suggest creating a module for callers with a helper method that creates a poolboy transaction and applies a Worker arg from the pool to a function. Example below cribbed from their github:

squery(PoolName, Sql) ->
    poolboy:transaction(PoolName, fun(Worker) ->
                                     gen_server:call(Worker, {squery, Sql})
                                  end).

That said, would Erlang RPC suit your needs better? Details on Erlang RPC at http://www.erlang.org/doc/man/rpc.html. A good treatment of Erlang RPC is found at http://learnyousomeerlang.com/distribunomicon#rpc.

这篇关于Erlang节点到节点消息吞吐量，超时和保证的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！