Spring批量远程分块和远程分区之间的区别 [英] Difference between spring batch remote chunking and remote partitioning

问题描述

spring批量远程分块和远程分区有什么区别？

What is the difference between spring batch remote chunking and remote partitioning?

我无法理解Spring批处理中远程分块和远程分区之间的区别。有人可以解释一下吗？

I can not understand the difference between remote chunking and remote partitioning in spring batch. Could anybody please explain?

推荐答案

远程分区

分区是主/从步骤配置，允许并行处理数据分区。通过一些元数据描述每个分区。例如，如果您正在处理数据库表，则分区1可能为0-100，分区2为101-200等。对于Spring Batch，主步骤使用 Partitioner 生成包含每个分区元数据的ExecutionContexts 。这些ExecutionContexts被分发到slave步骤，以便由 PartitionHandler （用于远程分区， MessageChannelPartitionHandler 。从属程序执行其步骤并返回由主服务器生成聚合的结果状态。

Partitioning is a master/slave step configuration that allows for partitions of data to be processed in parallel. Each partition is described via some metadata. For example, if you were processing a database table, partition 1 may be ids 0-100, partition 2 being 101-200, etc. For Spring Batch, a master step uses a Partitioner to generate ExecutionContexts that contain the metadata for each partition. These ExecutionContexts are distributed to slave step for processing by a PartitionHandler (for remote partitioning, the MessageChannelPartitionHandler is typically used). The slaves execute their step and return the resulting statuses for aggregation by the master.

有关远程分区的注意事项：

Things to note about remote partitioning:

• 输入和输出是从属设备的本地输出。例如，如果输入是文件，则从属设备需要访问该文件。


• 从属设备需要访问JobRepository。 Slaves是完全定义的Spring Batch步骤，所以他们需要JobRepository访问。

远程分块

远程分块类似于远程分区，因为它是主/从配置。但是，通过远程分块，数据由主设备读取并通过线路发送到从设备进行处理。处理完成后，ItemProcessor的结果将返回给master进行写入。

Remote chunking is similar to remote partitioning in that it is a master/slave configuration. However with remote chunking, the data is read at by the master and sent over the wire to the slave for processing. Once the processing is done, the result of the ItemProcessor is returned to the master for writing.

关于远程分块的注意事项：

Things to note about remote chunking:

• 所有I / O都由主服务器完成。


• 从服务器只处理处理，因此不需要JobRepository访问。


• 远程分块比远程分区更加I / O密集，因为实际数据是通过线路发送的，而不是描述它的元数据。

我做了关于扩展Spring Batch的演讲，并做了一个你可以在这里观看的远程分区演示： http://www.youtube.com/watch?v=CYTj5YT7CZU

I did a talk on scaling Spring Batch and do a demonstration of remote partitioning that you can watch here: http://www.youtube.com/watch?v=CYTj5YT7CZU

这篇关于Spring批量远程分块和远程分区之间的区别的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！