如何实现分布式处理 [英] How to implement distributed processing

问题描述

我请求进来为不同的样品（S1，S2，..）需要以线性方式进行处理（即，只有一个说在时刻S1-请求可以被处理）。我有工作人员服务，可以处理请求给予N个。我如何能实现RPC的队列模式，使得样品消耗一个在时间和仍然允许不同样本之间计算的分配？

我想与实现这一兔MQ因为它的简单性，群集功能，但我愿意考虑其他的解决方案也是如此。

下面是说明问题（有两名工人图片）

 工人1 
 + ----------- + 
 | | 
输入队列+ ----> | | ------- + 
 + -------------- + | | | | 
 | | | + ----------- + | 
 | S1，S2，S1，S1 | ------ + | 
 | | |工人2 | 
 + -------------- + | + ----------- + | 
 | | | | 
输出队列+ ----> | | ------- + 
 + -------------- + | | | 
 | | + ----------- + | 
 |（S1，S2，S1，S1）|&下;  -  + | 
 | | + ----------------------------- + 
 + -------------- + 
  

解决方案

一项简单的任务队列处理，假定任务保持不-intervening：

一个ZeroMQ对这个和复杂一点的设置>>>

智能讨论检查分割和放大器的正式行为模型设置;征服

（ 图-S：礼貌ZeroMQ / imatix 的）

在的 http://zguide.zeromq.org/page:all#Divide-and-Conquer

（只是为了一个灵感，还要检查与SIG_KILL扩展方式插件

）

---

的注：我没有（而 0 ）ZeroMQ隶属关系，与imatix相同。 然而，很多项目，由于这个神话般的ZeroMQ抽象和放聪明地工作也是后;建筑，恕我直言，我敢打赌，我可以说，这是一匹马力可能只从高性能，扩展能力，低延迟，分布式系统中受益。的

I have requests coming in for different samples (s1, s2, ..) that need to be processed in a linear fashion (i.e. only one say s1-request at the time can be processed). I have N-number of worker services that can process given requests. How can I implement rpc-queue pattern so that the samples are consumed one at the time and still allow distribution of calculation between different samples?

I would like to implement this with rabbit-mq because of it's simplicity, clustering capabilities, but I'm willing to consider other solutions as well.

Here is a picture to illustrate the problem ( with two workers)

                               worker 1 
                            +-----------+
                            |           |
 input queue          +---->|           |-------+
+--------------+      |     |           |       |
|              |      |     +-----------+       |
| s1,s2,s1,s1  |------+                         |
|              |      |        worker 2         |
+--------------+      |     +-----------+       |
                      |     |           |       |
 output queue         +---->|           |-------+
+--------------+            |           |       |
|              |            +-----------+       |
|(s1,s2,s1,s1) |<-+                             |
|              |  +-----------------------------+
+--------------+

解决方案

A trivial Task Queue processing, supposing tasks remain non-intervening:

A ZeroMQ has smart discussions for this and for a bit more complex setups >>>

Check a formal behaviour model setup for Divide & Conquer

(Fig-s: courtesy ZeroMQ/imatix)

at http://zguide.zeromq.org/page:all#Divide-and-Conquer

( Just for an inspiration, check also an extended approach with SIG_KILL add-on

)

---

n.b.: I have no ( rather Ø ) ZeroMQ affiliation, the same with imatix. However, after a lot of Projects, that work smart also due to this fabulous ZeroMQ-abstraction & architecture, IMHO I bet I can say, this is a horse-power one may only benefit from on high-performance, scale-able, low-latency, distributed systems.

这篇关于如何实现分布式处理的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！