Promise队列中的Bluebird-Queue并发无法按预期工作 [英] Bluebird-Queue Concurrency in Promise Queues not working as expected

问题描述

我正在通过NodeJS使用 bluebird-queue 将HTTP端点作为任务排队.每个任务都有3级Promise依赖，必须在完成之前解决.

I am using a bluebird-queue via NodeJS to queue HTTP endpoints as tasks. Each task has a 3-level Promise dependancy that has to resolve before it is complete.

一个任务

GET -> endpoint 1 // returns promise
    GET -> other endpoints in async // returns promise
        POST -> final endpoint // return promise

我用queue.add()将20,000个任务放入了蓝鸟队列，然后调用queue.start().捕获了所有错误，处理程序解析了Promise，因此任务可以完成.

I put 20,000 of these tasks into the bluebird-queue with queue.add() and then subsequently call queue.start(). All errors are caught and the handlers resolve the Promise so the task can complete.

我将并发设置为50.我最初的期望是，队列将在任何给定时间处理50，但是相反，它会等待前50个完成，然后再开始下一个50.

I have set concurrency to 50. My initial expectation is that the queue will process 50 at any given time but instead it waits for the first 50 to complete before starting on the next 50.

不幸的是，其中一些请求最多可能需要10秒钟才能完成-如果单个请求需要更长的时间才能完成，则整个队列都将停滞，直到Promise解决为止.

Unfortunately, some of these Requests can take up to 10 secs to complete - and if a single request takes longer to finish, the whole queue stalls until the Promise resolves.

如果这是预期的行为，我该怎么做/使用以确保队列中的任务在任何给定时间最多处理50个任务，而不是一次处理50个任务?

If this is the expected behaviour, what can I do/use to ensure that tasks in the queue processes a max of 50 tasks at any given time, instead of per 50 at a time?

这是我的配置设置:

var Queue = require('bluebird-queue'),
    queue = new Queue({
        concurrency: 50, 
        delay: 10, // ms
        interval: 1 // ms not quite sure what this means
    });

感谢您的帮助.

推荐答案

通读bluebird-queue的代码后，很明显，您所期望的关于并发的行为是所期望的.我同意这有点令人惊讶.似乎没有任何方法可以达到预期的效果.

After reading through the code for bluebird-queue, it seems clear that that the behavior you're seeing regarding concurrency is what is expected. I agree that this is somewhat surprising. There does not appear to be any way to get the desired behavior.

我建议尝试承诺队列.根据对代码的快速阅读，它似乎可以按您期望的那样工作.

I suggest trying promise-queue. Based on a quick reading of the code, it appears that it will work as you would expect.

这篇关于Promise队列中的Bluebird-Queue并发无法按预期工作的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！