如何使用bluebird并发选项用于映射函数 [英] How to use the bluebird concurrency option for the map function

问题描述

我想要检索名称列表，然后进行一些POST请求为每个名称。例如，我想要为每周的每一天的每个名称提出请求。但是，我需要限制并发POST请求的数量，因为目标服务器有速率限制。

  function getNames b $ b //打开mongodb连接
 //获取收集和名称数组
 //在promise中返回名称数组
} 
 
 function createDatesArray（） {
 //创建一个速率数组
 //在promise中返回日期数组
 // Ex。 return Promise.resolve（datesArray）; 
} 
 
 getNames（）。map（function（name）{
 return createDatesArray（）。map（function（date）{
 return requestData ）; 
}，{concurrency：5}）; 
}）then（function（）{
 // do something 
}）; 
  

这是使用蓝鸟并发的正确方法吗？

文档链接位于蓝鸟文档。

解决方案

简短答案：是，这会将请求数限制为5个。

警告：请记住，您可能仍然容易受到更多限制，例如HTTP客户端或您可能使用的任何其他池，模块和服务。

此外，Mongo连接意味着用作一个持久的连接，所以你可能应该只打开一个，然后使用它，而不是每次打开和关闭。

如果 createDatesArray 不执行任何异步操作，您不必Promise.resolve它， map 的静态变体为 Promise.map（datesArray，function（date）{...}）等。我也不会巢。假设 createDatesArray 确实是async：

  Promise.join（getNames ，createDatesArray（），function（names，dates）{
 var tasks = []; 
 names.forEach（function（name）{//创建名称的笛卡尔乘积*日期
日期。 forEach（function（date）{
 tasks.push（function（）{return requestData（name，date）;}）; 
}）; 
}）; 
 return Promise .map（tasks，function（job）{return job（）;}，{concurrency：5}）; 
}），然后（function（results）{
 // do whatever 
 }）; 
  

I am trying to use bluebird's map function with the built-in concurrency control.

I want to retrieve a list of names, then make a number of POST requests for each name. For example, I want to make a request for each name for each day of the week. However, I need to throttle the number of concurrent POST requests because the intended server has rate limits.

function getNames() {
    //Open mongodb connection
    //Get collection and array of names
    //return array of names in a promise 
}

function createDatesArray() {
    //Create an array of rates
    //return array of dates in a promise
    //Ex. return Promise.resolve(datesArray);
}

getNames().map(function (name) {
    return createDatesArray().map(function (date) {
        return requestData(date, name);
    }, {concurrency: 5});
}).then(function () {
//do something
});

Is this the correct way to use bluebird's concurrency?

The documentation link is here bluebird documentation.

解决方案

Short answer: yes , this will limit the number of requests to 5.

Caveat: keep in mind you might still be susceptible to more limits, such as the HTTP client's, or any other pools, modules and services you might be using.

Also, a Mongo connection is meant to be used as a persistent one, so you should probably only open one and then use it rather than open and close one every time.

If createDatesArray does not do anything asynchronous, you do not have to Promise.resolve it, you can instead use the static variant of map as Promise.map(datesArray, function(date){ ... }) etc. I would also not nest. Assuming createDatesArray is indeed async:

Promise.join(getNames(), createDatesArray(), function(names, dates){
    var tasks = [];
    names.forEach(function(name ){ // create Cartesian product of names * dates
        dates.forEach(function(date){
             tasks.push(function(){ return requestData(name, date); });
        });
    });
    return Promise.map(tasks, function(job){ return job(); } , { concurrency: 5} );
}), then(function(results){
     // do whatever
});

这篇关于如何使用bluebird并发选项用于映射函数的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！