Spring Batch:使用自定义批处理大小将列表写入数据库表 [英] Spring Batch : Write a List to a database table using a custom batch size

问题描述

背景

我有一个 Spring Batch 工作，其中:

1. FlatFileItemReader-一次从文件中读取一行
2. ItemProcesor-将文件中的行转换为List<MyObject>并返回List.也就是说，文件中的每一行都细分为List<MyObject>(文件中的1行转换为许多输出行).
3. ItemWriter-将List<MyObject>写入数据库表. (我使用了此 实现解压缩从处理器接收到的列表的实现 和delegae到JdbcBatchItemWriter)

1. FlatFileItemReader - Reads one row at a time from the file
2. ItemProcesor - Transforms the row from the file into a List<MyObject> and returns the List. That is, each row in the file is broken down into a List<MyObject> (1 row in file transformed to many output rows).
3. ItemWriter - Writes the List<MyObject> to a database table. (I used this implementation to unpack the list received from the processor and delegae to a JdbcBatchItemWriter)

问题

• 在第2点)，处理器可以返回100000个MyObject实例的List.
• 在点3)，委托JdbcBatchItemWriter将最终将带有100000个对象的整个List写入数据库.

• At point 2) The processor can return a List of 100000 MyObject instances.
• At point 3), The delegate JdbcBatchItemWriter will end up writing the entire List with 100000 objects to the database.

我的问题是:JdbcBatchItemWriter不允许自定义批处理大小.出于所有实际目的，该步骤的batch-size = commit-interval.考虑到这一点，在 Spring Batch 中是否提供了ItemWriter的另一种实现方式，它允许写入数据库并允许可配置的批处理大小?如果没有，该如何亲自写一名习惯作家来实现这一目标?

My question is : The JdbcBatchItemWriter does not allow a custom batch size. For all practical purposes, the batch-size = commit-interval for the step. With this in mind, is there another implementation of an ItemWriter available in Spring Batch that allows writing to the database and allows configurable batch size? If not, how do go about writing a custom writer myself to acheive this?

推荐答案

我看不到在JdbcBatchItemWriter上设置批处理大小的明显方法.但是，您可以扩展编写器并使用自定义BatchPreparedStatementSetter指定批处理大小.这是一个简单的示例:

I see no obvious way to set the batch size on the JdbcBatchItemWriter. However, you can extend the writer and use a custom BatchPreparedStatementSetter to specify the batch size. Here is a quick example:

public class MyCustomWriter<T> extends JdbcBatchItemWriter<T> {

    @Override
    public void write(List<? extends T> items) throws Exception {
        namedParameterJdbcTemplate.getJdbcOperations().batchUpdate("your sql", new BatchPreparedStatementSetter() {
            @Override
            public void setValues(PreparedStatement ps, int i) throws SQLException {
                // set values on your sql
            }

            @Override
            public int getBatchSize() {
                return items.size(); // or any other value you want
            }
        });
    }

}

The StagingItemWriter in the samples is an example of how to use a custom BatchPreparedStatementSetter as well.

这篇关于Spring Batch:使用自定义批处理大小将列表写入数据库表的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！