Spring Batch处理记录，但不将其插入数据库 [英] Spring Batch processes the records, but is not inserting them into the database

问题描述

问题

当我开始使用单独的线程来同时运行几次相同的作业时，正在发生的事情是没有插入必须从Writer中插入的记录(在处理完这些记录后)进入数据库.当我同时运行两组数据时，批处理将正确运行:

• 记录已处理的数据集1:3606(预期为3606).
• 记录已处理的dataSet2:1776(预期为1776).

从下图中可以看出，Spring Batch读取和写入的记录数符合预期:

上下文

在这个项目中，我使用MySQL作为数据库和Hibernate.

某些代码

• 批处理配置，作业和步骤

  @Configuration@EnableBatchProcessing公共类BatchConfig扩展了DefaultBatchConfigurer{@Autowired私有JobBuilderFactory jobBuilderFactory;@Autowired私有StepBuilderFactory stepBuilderFactory;@Autowired私有StepSkipListener stepSkipListener;@Autowired私有MainJobExecutionListener mainJobExecutionListener;@豆角，扁豆公共TaskExecutor taskExecutor(){ThreadPoolTask​​Executor taskExecutor = new ThreadPoolTask​​Executor();taskExecutor.setMaxPoolSize(10);taskExecutor.setThreadNamePrefix("batch-thread-")；返回taskExecutor;}@豆角，扁豆公共JobLauncher jobLauncher()引发异常{SimpleJobLauncher jobLauncher = new SimpleJobLauncher();jobLauncher.setJobRepository(getJobRepository());jobLauncher.setTaskExecutor(taskExecutor());jobLauncher.afterPropertiesSet();返回jobLauncher;}@豆角，扁豆public Step mainStep(ReaderImpl读取器，ProcessorImpl处理器，WriterImpl写入器){返回stepBuilderFactory.get("step").<列表< ExcelLoad> ;,发票> chunk(10).reader(阅读器).processor(处理器).writer(作家).faultTolerant().skipPolicy(new ExceptionSkipPolicy()).listener(stepSkipListener).建造();}@豆角，扁豆public Job mainJob(Step mainStep){返回jobBuilderFactory.get("mainJob").listener(mainJobExecutionListener).incrementer(新的RunIdIncrementer()).start(mainStep).建造();}} 

• 作家

  @Override公共无效写入(List< ;?扩展发票>列表){invoiceRepository.saveAll(list);} 

• 存储库

  @存储库公共接口InvoiceRepository扩展了JpaRepository< Invoice，Integer>.{} 

• 属性

  spring.main.allow-bean-definition-overriding = truespring.batch.initialize-schema =总是spring.batch.job.enabled = falsespring.datasource.driver-class-name = com.mysql.cj.jdbc.Driverspring.datasource.url = jdbc:mysql://localhost:3306/bd_dev?autoReconnect = true& useTimezone = true& useLegacyDatetimeCode = false& serverTimezone = Europe/Paris& zeroDateTimeBehavior = convertToNullspring.datasource.username = rootspring.datasource.password =密码spring.jpa.properties.hibernate.dialect = org.hibernate.dialect.MySQL5InnoDBDialect 

在使用单独的线程之前，已处理的记录已正确插入数据库中.可能会发生什么?

解决方案

在使用单独的线程之前，已处理的记录已正确插入数据库中.可能会发生什么?

如果决定使用多线程步骤，则需要确保批处理工件(读取器，写入器等)是线程安全的.根据您共享的内容，write方法在线程之间不同步，因此不是线程安全的.多线程文档的步骤部分.

您需要同步它(通过使用 synchronized 关键字或使用 Lock 等)或将您的编写器包装在 解决方案

Before using the separate threads, the processed records were inserted into the database correctly. What could be happening?

If you decide to use a multi-threaded step, you need to make sure your batch artefacts (reader, writer, etc) are thread-safe. From what you shared, the write method is not synchronized between threads and hence is not thread-safe. This is explained in the Multi-threaded Step section of the documentation.

You need to either synchronize it (by using the synchronized key word, or using a Lock, etc) or wrap your writer in a SynchronizedItemStreamWriter.

这篇关于Spring Batch处理记录，但不将其插入数据库的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！