Spring Boot 2升级-Spring Boot数据JPA saveAll()非常慢 [英] Spring boot 2 upgrade - spring boot data jpa saveAll() very slow

问题描述

我将mysql与hibernate和spring引导数据jpa一起使用(spring-boot-starter-data-jpa和mysql-connector-java).最近，我将Spring Boot项目从1.5升级到了2.0.弹簧数据CrudRepository的用于保存可迭代对象的API已将形式save()更改为saveAll().我在代码中进行了更改，并且可以运行，但是速度很慢:

I use mysql with hibernate and spring boot data jpa (spring-boot-starter-data-jpa and mysql-connector-java). Recently I upgraded my spring boot project from 1.5 to 2.0. The API of the spring data CrudRepository to save an iterable has changed form save() to saveAll(). I made the changes in the code and it works but it is very slow:

• 插入10项->慢2倍(49ms-> 95ms)
• 插入100项->慢6倍(132ms-> 840ms)
• 插入1000项->慢10倍(792ms-> 8028ms)
• 插入10000项->慢15倍(4912ms-> 73542ms)
• 插入100000项->慢22倍(32042ms-> 712702ms)

我已经用空表测试了两个弹簧版本的插件. mysql服务器版本未更改:5.7.21-MySQL社区服务器(GPL)

I have tested the inserts of both spring versions with an empty table. The mysql server version hasn't change: 5.7.21 - MySQL Community Server (GPL)

我每天需要插入约200万个项目，因此这种速度显着下降.这是我的配置:

I need to insert ~2M items daily so this slowdown is dramatically. This is my configuration:

spring.datasource.url = jdbc:mysql://localhost:3306/service?useSSL=false&rewriteBatchedStatements=true
spring.datasource.username = service
spring.datasource.password = service
spring.datasource.driver-class-name=com.mysql.jdbc.Driver
spring.datasource.hikari.maximum-pool-size=5

spring.jpa.properties.hibernate.dialect = org.hibernate.dialect.MySQL5InnoDBDialect
spring.jpa.properties.hibernate.jdbc.batch_size=50
spring.jpa.properties.hibernate.order_inserts=true
spring.jpa.properties.hibernate.order_updates=true
spring.jpa.properties.hibernate.jdbc.batch_versioned_data=true

有人知道更新有什么变化以及如何再次加快更新速度吗?

Does anybody know what changed with the update and how to speed it up again?

推荐答案

在application.properties中设置spring.jpa.properties.hibernate.generate_statistics = true

in application.properties set spring.jpa.properties.hibernate.generate_statistics = true

hibernate:
  generate_statistics: true

https://www.baeldung.com/spring-data-jpa -批量插入

这篇关于Spring Boot 2升级-Spring Boot数据JPA saveAll()非常慢的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！