如何控制并行Spring Batch作业的数量 [英] How to control the number of parallel Spring Batch jobs

问题描述

我有一个生成报告的应用程序.由于此类报告的准备工作非常繁琐，因此它们与Spring Batch异步准备.对于此类报告的请求是使用HTTP通过REST接口创建的.

I have a report generating application. As preparation of such reports is heavyweight, they are prepared asynchronously with Spring Batch. Requests for such reports are created via REST interface using HTTP.

目标是REST资源仅将报告执行排队并完成(

The goal is that the REST resource simply queues report execution and completes (as described in documentation). Thus a TaskExecutor has been provided for the JobLauncher:

    <bean id="jobLauncher" class="org.springframework.batch.core.launch.support.SimpleJobLauncher">
        <property name="jobRepository" ref="jobRepository" />
        <property name="taskExecutor">
        <bean class="org.springframework.core.task.SimpleAsyncTaskExecutor"/>
    </property>
</bean>

由于报告确实很重要，因此在给定时间只能生成指定数量的报告.希望能够配置Spring Batch一次仅产生2个实例，已指定 concurrencyLimit :

As the reports are really heavyweight, only a specified number of them can be produced at a given time. Hoping to be able to configure Spring Batch to produce 2 instances at a time only, concurrencyLimit has been specified:

    <bean id="jobLauncher" class="org.springframework.batch.core.launch.support.SimpleJobLauncher">
        <property name="jobRepository" ref="jobRepository" />
        <property name="taskExecutor">
        <bean class="org.springframework.core.task.SimpleAsyncTaskExecutor">
            <property name="concurrencyLimit" value="2" />
        </bean>
    </property>
</bean>

不幸的是，当已经有2个作业在运行时，启动作业调用被阻止: jobLauncher.run(job，builder.toJobParameters());

Unfortunately, when 2 jobs are already running, the launch job call is blocked: jobLauncher.run(job, builder.toJobParameters());

显然，jobLauncher立即尝试执行作业.我想它会在线程可用时立即将作业排队等待执行.这样，我可以通过简单地添加其他处理实例来扩展我的应用程序，所有这些实例都使用相同的作业存储库数据库.

Apparently jobLauncher immediately attempts to execute job. I would imagine it rather queue job for execution as soon as a thread is available. This way I could scale my application by simply adding additional processing instances, all using the same job repository database.

类似的问题是在这里提出的.我将开始探索春季批处理集成但我不确定这是否是正确的方向.

Similar question was asked here. I'm about to start exploring Spring Batch Integration but I'm not sure if that's the right direction.

我的用例对我来说似乎并不罕见，是否应该针对我显然找不到的用例进行广泛讨论?

My usecase does not seem that uncommon to me, should't there be a widely discussed pattern for it that I am apparently unable to find?

谢谢 f

推荐答案

SimpleAsyncTaskExecutor，因为它会为每个任务生成一个新线程.它还不支持更强大的概念，例如线程池和任务排队.

SimpleAsyncTaskExecutor isn't recommended for heavy use since it spawns a new thread with each task. It also does not support more robust concepts like thread pooling and queueing of tasks.

如果您看一下ThreadPoolTaskExecutor，它支持更健壮的任务执行范例，其中包括任务排队和使用线程池，而不是生成随机的，未重用的线程.

If you take a look at the ThreadPoolTaskExecutor, it supports a more robust task execution paradigm with things like queueing of tasks and using a thread pool instead of spawning random, un-reused threads.

您可以在以下Javadoc中阅读有关ThreadPoolTaskExecutor的更多信息:

You can read more about the ThreadPoolTaskExecutor in the javadoc here: http://docs.spring.io/spring/docs/current/javadoc-api/org/springframework/scheduling/concurrent/ThreadPoolTaskExecutor.html

这篇关于如何控制并行Spring Batch作业的数量的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！