在 node.js 中协调并行执行 [英] Coordinating parallel execution in node.js

问题描述

node.js 的事件驱动编程模型使得协调程序流程有些棘手.

The event-driven programming model of node.js makes it somewhat tricky to coordinate the program flow.

简单的顺序执行变成了嵌套的回调，这很容易(虽然写下来有点复杂).

Simple sequential execution gets turned into nested callbacks, which is easy enough (though a bit convoluted to write down).

但是并行执行呢?假设您有三个可以并行运行的任务 A、B、C，当它们完成后，您想将它们的结果发送到任务 D.

But how about parallel execution? Say you have three tasks A,B,C that can run in parallel and when they are done, you want to send their results to task D.

使用 fork/join 模型，这将是

With a fork/join model this would be

• 叉A
• 叉 B
• 叉C
• 加入A、B、C，运行D

我如何在 node.js 中编写它?是否有任何最佳实践或食谱?我是否必须每次都手动解决方案，或者是否有一些库这方面的帮手?

How do I write that in node.js ? Are there any best practices or cookbooks? Do I have to hand-roll a solution every time, or is there some library with helpers for this?

推荐答案

node.js 中没有真正的并行，因为它是单线程的.但是，可以按您无法事先确定的顺序安排和运行多个事件.像数据库访问这样的一些事情实际上是并行的"，因为数据库查询本身在单独的线程中运行，但在完成后重新集成到事件流中.

Nothing is truly parallel in node.js since it is single threaded. However, multiple events can be scheduled and run in a sequence you can't determine beforehand. And some things like database access are actually "parallel" in that the database queries themselves are run in separate threads but are re-integrated into the event stream when completed.

那么，您如何在多个事件处理程序上安排回调?嗯，这是浏览器端 javascript 动画中常用的一种技术:使用变量来跟踪完成.

So, how do you schedule a callback on multiple event handlers? Well, this is one common technique used in animations in browser side javascript: use a variable to track the completion.

这听起来像是一种黑客行为，确实如此，而且在进行跟踪时留下一堆全局变量并且使用较少的语言，这听起来可能很混乱.但是在 javascript 中我们可以使用闭包:

This sounds like a hack and it is, and it sounds potentially messy leaving a bunch of global variables around doing the tracking and in a lesser language it would be. But in javascript we can use closures:

function fork (async_calls, shared_callback) {
  var counter = async_calls.length;
  var callback = function () {
    counter --;
    if (counter == 0) {
      shared_callback()
    }
  }

  for (var i=0;i<async_calls.length;i++) {
    async_calls[i](callback);
  }
}

// usage:
fork([A,B,C],D);

在上面的例子中，我们通过假设异步和回调函数不需要参数来保持代码简单.您当然可以修改代码以将参数传递给异步函数，并让回调函数累积结果并将其传递给 shared_callback 函数.

In the example above we keep the code simple by assuming the async and callback functions require no arguments. You can of course modify the code to pass arguments to the async functions and have the callback function accumulate results and pass it to the shared_callback function.

实际上，即使如此，fork() 函数已经可以使用闭包将参数传递给异步函数:

Actually, even as is, that fork() function can already pass arguments to the async functions using a closure:

fork([
  function(callback){ A(1,2,callback) },
  function(callback){ B(1,callback) },
  function(callback){ C(1,2,callback) }
],D);

剩下要做的就是将 A、B、C 的结果累加并传递给 D.

the only thing left to do is to accumulate the results from A,B,C and pass them on to D.

我无法抗拒.吃早餐的时候一直在想这个.这是累积结果的 fork() 实现(通常作为参数传递给回调函数):

I couldn't resist. Kept thinking about this during breakfast. Here's an implementation of fork() that accumulates results (usually passed as arguments to the callback function):

function fork (async_calls, shared_callback) {
  var counter = async_calls.length;
  var all_results = [];
  function makeCallback (index) {
    return function () {
      counter --;
      var results = [];
      // we use the arguments object here because some callbacks 
      // in Node pass in multiple arguments as result.
      for (var i=0;i<arguments.length;i++) {
        results.push(arguments[i]);
      }
      all_results[index] = results;
      if (counter == 0) {
        shared_callback(all_results);
      }
    }
  }

  for (var i=0;i<async_calls.length;i++) {
    async_calls[i](makeCallback(i));
  }
}

这很容易.这使得 fork() 具有相当的通用性，可用于同步多个非同类事件.

That was easy enough. This makes fork() fairly general purpose and can be used to synchronize multiple non-homogeneous events.

Node.js 中的示例用法:

Example usage in Node.js:

// Read 3 files in parallel and process them together:

function A (c){ fs.readFile('file1',c) };
function B (c){ fs.readFile('file2',c) };
function C (c){ fs.readFile('file3',c) };
function D (result) {
  file1data = result[0][1];
  file2data = result[1][1];
  file3data = result[2][1];

  // process the files together here
}

fork([A,B,C],D);

<小时>

更新

此代码是在诸如 async.js 之类的库或各种基于 Promise 的库存在之前编写的.我想相信 async.js 受到了这个启发，但我没有任何证据.不管怎样..如果你今天想这样做，看看 async.js 或 promises.只需考虑上面的答案就可以很好地解释/说明 async.parallel 之类的事情是如何工作的.

---

Update

This code was written before the existence of libraries like async.js or the various promise based libraries. I'd like to believe that async.js was inspired by this but I don't have any proof of it. Anyway.. if you're thinking of doing this today take a look at async.js or promises. Just consider the answer above a good explanation/illustration of how things like async.parallel work.

为了完整起见，以下是您将如何使用 async.parallel:

For completeness sake the following is how you'd do it with async.parallel:

var async = require('async');

async.parallel([A,B,C],D);

请注意，async.parallel 的工作原理与我们上面实现的 fork 函数完全相同.主要区别在于它根据 node.js 约定将错误作为第一个参数传递给 D，将回调作为第二个参数传递.

Note that async.parallel works exactly the same as the fork function we implemented above. The main difference is it passes an error as the first argument to D and the callback as the second argument as per node.js convention.

使用 Promise，我们将其编写如下:

Using promises, we'd write it as follows:

// Assuming A, B & C return a promise instead of accepting a callback

Promise.all([A,B,C]).then(D);

这篇关于在 node.js 中协调并行执行的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！