如何使用 JdbcCursorItemReader 和多线程从 Oracle Db 读取数据 [英] How to read data from Oracle Db using JdbcCursorItemReader with Multi threading

问题描述

我有 spring 批处理应用程序并配置了这样的步骤:

ThreadPoolTask​​Executor taskExecutor = new ThreadPoolTask​​Executor();taskExecutor.setCorePoolSize(4);taskExecutor.setMaxPoolSize(10);taskExecutor.afterPropertiesSet();返回 this.stepBuilderFactory.get("step1").<我的模型，我的模型>块(2500).读者(读者()).作家(作家()).taskExecutor(taskExecutor).build();

和这样的读者:

@Beanpublic JdbcCursorItemReader读者(){JdbcCursorItemReaderreader = new JdbcCursorItemReader();reader.setDataSource(dataSource);reader.setSql("select * from User");reader.setRowMapper(new BeanPropertyRowMapper<>(Mymodel.class));reader.setVerifyCursorPosition(false);返回读者；}

当我执行应用程序时，出现此错误:

<块引用>

org.springframework.jdbc.UncategorizedSQLException:尝试处理下一个行失败；SQL [select * from User] 的未分类 SQLException；SQL状态[99999]；错误代码 [17289];最后一行之后的结果集；嵌套异常是 java.sql.SQLException:最后一行之后的结果集

你能帮我解决这个问题吗.

解决方案

JdbcCursorItemReader 是 不是线程安全的，因为它包装了一个不是线程安全的 ResultSet 对象.>

您可以使用 JdbcPagingItemReader 其中 是线程安全的，并且可以选择配置页面大小以匹配块大小，以便在同一线程中处理每个页面.

以下答案中的更多详细信息:https://stackoverflow.com/a/28724199/5019386.>

希望这会有所帮助.

I had spring batch application and configured step like this:

ThreadPoolTaskExecutor taskExecutor = new ThreadPoolTaskExecutor();
taskExecutor.setCorePoolSize(4);
taskExecutor.setMaxPoolSize(10);
taskExecutor.afterPropertiesSet();
return this.stepBuilderFactory.get("step1")
            .<Mymodel, Mymodel>chunk(2500)              
            .reader(reader())
            .writer(writer())
            .taskExecutor(taskExecutor)
            .build();

And reader like this:

@Bean
public JdbcCursorItemReader<Mymodel> reader() {
    JdbcCursorItemReader<Mymodel> reader = new JdbcCursorItemReader<Mymodel>();
    reader.setDataSource(dataSource);
    reader.setSql("select * from User");
    reader.setRowMapper(new BeanPropertyRowMapper<>(Mymodel.class));
    reader.setVerifyCursorPosition(false);
    return reader;
}

When I execute application , getting this error:

org.springframework.jdbc.UncategorizedSQLException: Attempt to process next row failed; uncategorized SQLException for SQL [select * from User]; SQL state [99999]; error code [17289]; Result set after last row; nested exception is java.sql.SQLException: Result set after last row

Can you please help me to solve this.

解决方案

The JdbcCursorItemReader is not thread-safe as it wraps a ResultSet object which is not thread safe.

You can use a JdbcPagingItemReader which is thread safe and optionally configure the page size to match the chunk size so that each page is processed in the same thread.

More details in the following answer: https://stackoverflow.com/a/28724199/5019386.

Hope this helps.

这篇关于如何使用 JdbcCursorItemReader 和多线程从 Oracle Db 读取数据的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！