在 Javascript 中以 50 个小块执行 100K Promises [英] Executing 100K Promises in Javascript in small 50 Chunks
问题描述
我有一个函数可以对服务进行 REST 调用并返回一个承诺.让我们调用该函数 Execute().该函数接受一个 ID 并将该 ID 作为 GET 参数发送到 REST 端点，该端点将 ID 与一些附加信息一起保存在 mongoDB 数据库中.
I have a function that makes a REST call to a service and returns a promise. lets call that function Execute(). The function takes an ID and sends the ID as a GET parameter to a REST end point which persists the ID in a mongoDB db with some additional info.
在客户端中，我需要从 ID(0 到 100k)运行执行"100k 次并显示每个状态(成功还是失败).
In the client, I will need to run "Execute" 100k times from IDs (0 to 100k) and show the status of each (Whether succeeded or failed).
我做了很明显的事情，我创建了一个从 0 到 100k 的循环并运行执行传递i.这导致我的 Chrome 最终冻结内存不足(资源不足).它还导致所有其余调用的网络拥塞在后端.
I did the obvious and I created a loop from 0 to 100k and run execute passing "i. That caused my Chrome to eventually freeze running out of memory (insufficient resources). It also caused network congestion from all the rest calls going at the back end.
所以我想将这 10 万个砍"成可管理的数量，例如每个 50 个承诺调用.当这 50 个都完成后(无论是失败还是成功)，我想使用 Promise.all([]).then 执行下一个 50，直到所有 100k 完成.这样我同时控制了网络拥塞和内存.但是，我似乎不知道如何解决这个问题.这是我的代码.
So I wanted to "chop" those 100k into manageable amount like 50 promises call each. And when those 50 are all done (whether failed or succeeded) I want to use Promise.all([]).then execute the next 50 until all the 100k are done. This way I control the network congestion and memory at the same time. However I can't seem to know how to shake this down. Here is my code.
let promises = []
for (let i = 0; i < 100000, i++)
{
promises.push(execute(i))
if (i % 50 === 0)
{
Promise.all(promises)
.then (a => updateStatus (a, true))
.catch (a => updateStatus (a, false))
}
}
Javascript 的异步特性将继续执行循环的其余部分并继续执行.我真的不想每 50 次迭代就设置一个计时器来保持循环，因为这会阻塞 UI 并使我的应用程序同步.关于我如何解决这个问题有什么建议吗?
The asynchronous nature of Javascript will keep executing the rest of the loop and executing. I really don't want to put a timer to hold the loop every 50 iterations because this will block the UI and kind of turned my app synchronous. Any suggestions as to how I tackle this?
非常感谢.
Javascript 新手.
New to Javascript.
推荐答案
对于 promises 本身，这只能通过递归来完成.
With promises itself, this can be done only with recursion.
如果你可以使用新版本的 Node.js，使用 async await，它会按你的预期工作，那么你可以使用 await Promise.all(promises)
If you can use new version of Node.js, use async await, it will work as you expect, then you can use await Promise.all(promises)
如果你不能，那么有一个很好的库，可以用这个方法一次执行 50 个异步调用:https://caolan.github.io/async/docs.html#parallelLimit
If you cant, then there is nice library that can execute 50 asynchronous calls at once with this method: https://caolan.github.io/async/docs.html#parallelLimit
它甚至比块更好，因为如果块中有 1 个慢速回调，它会阻塞其他一切.在并行限制下，它只是一直执行 50 个回调.(但是，如果您坚持使用 .series 方法，则可以预先创建 50 个块)
It is even better than chunks, because if you have 1 slow callback in chunk, it will block everything else. With parallel limit, it just keep executing 50 callbacks all the time. (however you can just pre-create chunks by 50 if you insist on them and use .series method)
这篇关于在 Javascript 中以 50 个小块执行 100K Promises的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
