协调node.js中的并行执行 [英] Coordinating parallel execution in node.js

问题描述

node.js的事件驱动编程模型使得协调程序流有点棘手。

The event-driven programming model of node.js makes it somewhat tricky to coordinate the program flow.

简单的顺序执行变成嵌套的回调，很容易足够（虽然有点复杂的写下来）。

Simple sequential execution gets turned into nested callbacks, which is easy enough (though a bit convoluted to write down).

但是平行执行怎么样？假设你有三个任务A，B，C可以并行运行，当他们完成后，你想发送他们的结果到任务D。

But how about parallel execution? Say you have three tasks A,B,C that can run in parallel and when they are done, you want to send their results to task D.

加入模型

• fork A


• fork B


• fork C


• 加入A，B，C，运行D

• fork A
• fork B
• fork C
• join A,B,C, run D

如何我写在node.js？有任何最佳做法或食谱吗？我每次都必须手动滚动解决方案，还是有一些库帮助这个？

How do I write that in node.js ? Are there any best practices or cookbooks? Do I have to hand-roll a solution every time, or is there some library with helpers for this?

推荐答案

node.js中没有什么是真正的并行，因为它是单线程的。但是，可以按照您无法预先确定的顺序安排和运行多个事件。有些事情，比如数据库访问实际上是并行，因为数据库查询本身是在单独的线程中运行，但是在完成后重新集成到事件流中。

Nothing is truly parallel in node.js since it is single threaded. However, multiple events can be scheduled and run in a sequence you can't determine beforehand. And some things like database access are actually "parallel" in that the database queries themselves are run in separate threads but are re-integrated into the event stream when completed.

，你如何安排多个事件处理程序的回调？好吧，这是一个常用的技术，在浏览器边的动画javascript：使用一个变量跟踪完成。

So, how do you schedule a callback on multiple event handlers? Well, this is one common technique used in animations in browser side javascript: use a variable to track the completion.

这听起来像一个黑客，潜在的混乱留下一堆全局变量围绕做跟踪和在较小的语言，它将是。但在JavaScript中我们可以使用闭包：

This sounds like a hack and it is, and it sounds potentially messy leaving a bunch of global variables around doing the tracking and in a lesser language it would be. But in javascript we can use closures:

function fork (async_calls, shared_callback) {
  var counter = async_calls.length;
  var callback = function () {
    counter --;
    if (counter == 0) {
      shared_callback()
    }
  }

  for (var i=0;i<async_calls.length;i++) {
    async_calls[i](callback);
  }
}

// usage:
fork([A,B,C],D);

在上面的例子中，我们通过假设async和callback函数不需要任何参数来保持代码简单。您当然可以修改代码以将参数传递给异步函数，并让回调函数累积结果并将其传递给shared_callback函数。

In the example above we keep the code simple by assuming the async and callback functions require no arguments. You can of course modify the code to pass arguments to the async functions and have the callback function accumulate results and pass it to the shared_callback function.

实际上，即使是 fork（）

fork([
  function(callback){ A(1,2,callback) },
  function(callback){ B(1,callback) },
  function(callback){ C(1,2,callback) }
],D);

唯一剩下要做的就是累积A，B，C的结果，到D。

the only thing left to do is to accumulate the results from A,B,C and pass them on to D.

无法抗拒。在早餐期间考虑这一点。这里是一个实现 fork（），它累积结果（通常作为参数传递给回调函数）：

I couldn't resist. Kept thinking about this during breakfast. Here's an implementation of fork() that accumulates results (usually passed as arguments to the callback function):

function fork (async_calls, shared_callback) {
  var counter = async_calls.length;
  var all_results = [];
  function makeCallback (index) {
    return function () {
      counter --;
      var results = [];
      // we use the arguments object here because some callbacks 
      // in Node pass in multiple arguments as result.
      for (var i=0;i<arguments.length;i++) {
        results.push(arguments[i]);
      }
      all_results[index] = results;
      if (counter == 0) {
        shared_callback(all_results);
      }
    }
  }

  for (var i=0;i<async_calls.length;i++) {
    async_calls[i](makeCallback(i));
  }
}

这使得 fork（）相当通用，可用于同步多个非同类事件。

That was easy enough. This makes fork() fairly general purpose and can be used to synchronize multiple non-homogeneous events.

在Node.js中：

// Read 3 files in parallel and process them together:

function A (c){ fs.readFile('file1',c) };
function B (c){ fs.readFile('file2',c) };
function C (c){ fs.readFile('file3',c) };
function D (result) {
  file1data = result[0][1];
  file2data = result[1][1];
  file3data = result[2][1];

  // process the files together here
}

fork([A,B,C],D);

---

更新

这段代码是在存在像async.js或各种基于promise的库之前编写的。我想相信async.js的灵感来自这个，但我没有任何证据。反正..如果你想这样做今天看看async.js或承诺。只要考虑上面的答案一个很好的解释/插图如何async.parallel工作。

---

Update

This code was written before the existence of libraries like async.js or the various promise based libraries. I'd like to believe that async.js was inspired by this but I don't have any proof of it. Anyway.. if you're thinking of doing this today take a look at async.js or promises. Just consider the answer above a good explanation/illustration of how things like async.parallel work.

这篇关于协调node.js中的并行执行的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！