在Spring Integration中需要并行处理多个文件 [英] Need to process multiple files in parallel in Spring Integration

问题描述

我有一个SFTP目录，可以读取文件并将文件发送给ServiceActivator进行进一步处理.在任何时候，我都需要使用处理程序并行处理它们.

I have a SFTP directory and reading files and sending the files for further processing to a ServiceActivator.At any point I need to process them parallely using the handler.

这是我的SPring Integration Java DSL流程.

Here is my SPring Integration java DSL flow.

IntegrationFlows.from(Sftp.inboundAdapter(getSftpSessionFactory())
                        .temporaryFileSuffix("COPY")
                        .localDirectory(directory)
                        .deleteRemoteFiles(false)
                        .preserveTimestamp(true)
                        .remoteDirectory("remoteDir"))
                        .patternFilter("*.txt")), e -> e.poller(Pollers.fixedDelay(500).maxMessagesPerPoll(5)))
                        .handle("mybean", "myMethod")
                        .handle(Files.outboundAdapter(new File("success")))         
                        .deleteSourceFiles(true)
                        .autoCreateDirectory(true))
                        .get();

更新:这是我的ThreadPoolExecutor:

Update:Here is my ThreadPoolExecutor:

@Bean(name = "executor")
public Executor getExecutor()
{
    ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
    executor.setCorePoolSize(4);
    executor.setMaxPoolSize(4);
    executor.setQueueCapacity(20);          
    executor.initialize();
    return executor;
}

推荐答案

无论如何，Sftp.inboundAdapter()(SftpInboundFileSynchronizingMessageSource)会一一返回远程文件.首先，它将它们同步到本地目录，然后才轮询它们以作为File有效负载进行消息处理.

The Sftp.inboundAdapter() (SftpInboundFileSynchronizingMessageSource) returns remote files one by one anyway. First of all it synchronizes them to the local directory and only after that poll them for message processing as File payload.

要并行处理它们，仅需在e.poller()定义中添加taskExecutor就足够了，所有这些maxMessagesPerPoll(5)都将分配给不同的线程.

To process them in parallel that would be just enough to add a taskExecutor to your e.poller() definition and all those maxMessagesPerPoll(5) will be distributed to different threads.

这篇关于在Spring Integration中需要并行处理多个文件的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！