使用ES6的Promise.all()时限制并发的最佳方法是什么? [英] What is the best way to limit concurrency when using ES6's Promise.all()?

问题描述

我有一些代码可以遍历从数据库中查询出来的列表，并对该列表中的每个元素进行HTTP请求.该列表有时可能是一个相当大的数目(成千上万个)，并且我想确保我不会遇到具有成千上万个并发HTTP请求的Web服务器.

I have some code that is iterating over a list that was queried out of a database and making an HTTP request for each element in that list. That list can sometimes be a reasonably large number (in the thousands), and I would like to make sure I am not hitting a web server with thousands of concurrent HTTP requests.

此代码的缩写版本目前看起来像这样...

An abbreviated version of this code currently looks something like this...

function getCounts() {
  return users.map(user => {
    return new Promise(resolve => {
      remoteServer.getCount(user) // makes an HTTP request
      .then(() => {
        /* snip */
        resolve();
      });
    });
  });
}

Promise.all(getCounts()).then(() => { /* snip */});

此代码在节点4.3.2上运行.重申一下，可以管理Promise.all以便在任何给定时间仅进行一定数量的Promises吗?

This code is running on Node 4.3.2. To reiterate, can Promise.all be managed so that only a certain number of Promises are in progress at any given time?

推荐答案

请注意，Promise.all()不会触发诺言开始工作，而创建诺言本身会触发.

Note that Promise.all() doesn't trigger the promises to start their work, creating the promise itself does.

牢记这一点，一种解决方案是检查承诺何时得到解决，是否应该启动新的承诺，或者您是否已经达到极限.

With that in mind, one solution would be to check whenever a promise is resolved whether a new promise should be started or whether you're already at the limit.

但是，这里确实没有必要重新发明轮子. 为此目的可以使用的一个库是es6-promise-pool .从他们的例子中:

However, there is really no need to reinvent the wheel here. One library that you could use for this purpose is es6-promise-pool. From their examples:

// On the Web, leave out this line and use the script tag above instead. 
var PromisePool = require('es6-promise-pool')

var promiseProducer = function () {
  // Your code goes here. 
  // If there is work left to be done, return the next work item as a promise. 
  // Otherwise, return null to indicate that all promises have been created. 
  // Scroll down for an example. 
}

// The number of promises to process simultaneously. 
var concurrency = 3

// Create a pool. 
var pool = new PromisePool(promiseProducer, concurrency)

// Start the pool. 
var poolPromise = pool.start()

// Wait for the pool to settle. 
poolPromise.then(function () {
  console.log('All promises fulfilled')
}, function (error) {
  console.log('Some promise rejected: ' + error.message)
})

这篇关于使用ES6的Promise.all()时限制并发的最佳方法是什么?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！