顺序执行许多承诺(概念) [英] Execute many promises sequentially (Concept)

问题描述

(我的目标是阐明我对此问题的概念，而不是代码)

(My target is clarify my concept about the problem, not code)

我想按顺序执行一个promise数组，但是nodeJS抛出一个关于许多并行执行的promise的奇怪错误(因为我将数组限制为20个promise和works，50个promise and works，但是9000个promise and explode.)

I want execute a array of promises sequentially, but nodeJS throw a strange error about many promises executed in parallel.(Because i limited that array to 20 promises and works, 50 promises and works, but 9000 promises and explode..)

• 我知道我们有一些解决方案，例如array.reduce()，循环等
• 我了解诺言状态(我的数组最初有未完成的诺言)

我的问题:我可以执行20个Promise，然后再执行20个Promise，依此类推，但是... 如果我按顺序执行我的Promise，nodeJS必须成功执行9k个Promise?我的观念不好吗?我的代码是错误的吗?

My question: I can execute 20 promises, then another 20 promises, etc but... If im executing my promises sequentially, nodeJS must execute 9k promises without problem? I have a bad concept? My code is wrong?

(令人怀疑，因为nodeJS在开始兑现承诺之前要等待一段时间)

(Im doubting because nodeJS wait some time before begin to resolve the promises)

我的情况:我尝试下载9k +图像(使用axios)，然后保存每个图像，然后依次等待5秒钟. [下载1张图像，保存该图像，等待5秒钟，然后下载下一张图像，保存..，等待...，等等.]可能吗?

My case: i trying download 9k+ images (with axios), then save each one and then wait 5 seconds sequentially. [download 1 image, save that image, wait 5 seconds, then download next image, save.., wait..., etc.. ] Possible?

推荐答案

我应该使用工作者池，而不是每次执行20次批处理，您总是会在等待下一个批处理开始之前最终等待最后一个完成，而应该设置您要连续下载多少次的限制这样做，您的承诺不超过20个，而且没有9000个长链

I would have used something like a worker pool instead of executing things in a batch of 20 each time, you will always end up waiting for the last one to finish before you start next 20 batch, instead you should set a limit of how many continious download you want to do so you have no more then 20 promises and not a long chain of 9000

同样的事情也可以通过迭代器来完成. (可以将同一迭代器传递给不同的工作人员，并且当某人调用第一个项目时，下一个工作人员将始终获得下一个工作人员)

The same thing can be accomplish with iterators also. (a same iterator can be passed to different workers and while someone calls the first item the next worker will always get the next one)

所以在零依赖的情况下，我会做这样的事情:

So with zero dependencies i would do something like this:

const sleep = n => new Promise(rs => setTimeout(rs, 1000))

async function sequentialDownload(iterator) {
  for (let [index, url] of iterator) {
    // figure out where to save the file
    const path = path.resolve(__dirname, 'images', index + '.jpg')
    // download all images as a stream
    const res = await axios.get(index, { responseType: 'stream' })

    // pipe the stream to disc
    const writer = fs.createWriteStream(path)
    res.data.pipe(writer)

    // wait for the download to complete
    await new Promise(resolve => writer.on('finish', resolve))
    // wait a extra 5 sec
    await sleep(5000)
  }
}

const arr = [url1, url2, url3] // to be downloaded
const workers = new Array(20) // create 20 "workers"
  .fill(arr.entries()) // fill it with same iterator
  .map(sequentialDownload) // start working

Promise.all(workers).then(() => {
  console.log('done downloading everything')
})

这篇关于顺序执行许多承诺(概念)的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！