使用 Spring App 中的期货处理并行异步操作 [英] Handle parallel async operations with futures in Spring App

查看:48
本文介绍了使用 Spring App 中的期货处理并行异步操作的处理方法,对大家解决问题具有一定的参考价值,需要的朋友们下面随着小编来一起学习吧!

问题描述

我有一个包含 +1M 条目的数据库,我正在以块的形式查询数据库(假设 chunkSize = 1000).

I have a db with +1M entries in it, I am querying the DB in chunks (lets say chunkSize = 1000).

迭代这个 1000 块,我想为每个实体向rabbitMQ 发布一条消息,将这个块的所有期货收集到一个数组/列表中.

Iterating over this chunk of 1000, I want to publish a message to rabbitMQ for each entity, collect all the futures for this chunk into an array/list.

等待这个futures数组,直到所有futures都完成/失败,然后把这个1000的数组过滤成两个新的list(successList,failureList),然后为mongoDB准备两个bulkUpdate操作,执行两个ops.

Wait on this futures array until all futures are completed/failed, then filter this array of 1000 into two new lists (successList, failureList), then prepare two bulkUpdate operations for mongoDB, and execute the two ops.

我必须逐块等待,否则,下一次迭代(用于获取数据库实体)将重新读取已经在进行中的实体.

I have to wait chunk by chunk, because otherwise, the next iteration (for fetching db entities) will re-read already in progress entities.

PS:我最初的假设是在列表中使用 CompletableFutures(我会在 Javascript 中做同样的事情,不确定 java 是否有其他方法)

PS: My initial assumption was to use CompletableFutures in a list (the same thing I would have done in Javascript, not sure if java has any other approach)

推荐答案

这个答案中所述 Spring AMQP 允许你等待 CorrelationData 中的一个 Future 来获取发布消息的结果.您无需发明自己的机制.

As explained in this answer Spring AMQP allows you to wait on a Future in CorrelationData to get the result of publishing a message. You don't need to invent your own mechanism.

如果您想存储有关发送的更多信息,您始终可以将 CorrelationData 子类化.

You can always subclass CorrelationData if you want to store more information about a send.

这篇关于使用 Spring App 中的期货处理并行异步操作的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!

查看全文
登录 关闭
扫码关注1秒登录
发送“验证码”获取 | 15天全站免登陆