JPA / Hibernate提高批量插入性能 [英] JPA/Hibernate improve batch insert performance

问题描述

我有一个数据模型，在ONE实体和其他11个实体之间有一对多的关系。这12个实体一起代表一个数据包。我遇到的问题是在这些关系的多方面发生的插入次数。其中一些可以有多达100个单独的值，因此为了将一个完整的数据包保存在数据库中，它需要多达500个插入。

我在InnoDB表中使用MySQL 5.5。现在，通过测试数据库，我发现在处理批量插入时可以轻松地执行每秒15000次插入操作（甚至可以使用LOAD DATA，但对于这种情况，这并不实际）。

有没有办法将这些单独的500个插入插入到5个使用Hibernate的VALUES（对于每个有100个值的链接实体）插入？

$ p
$ b

  @OneToMany（mappedBy =beat，cascade = CascadeType.ALL）
 @OrderBy（毫秒ASC）
 public List< AmbientLight>灯; 
  

我应该也提到一个重要信息 - 我正在使用 Play！ Framework 1.2.3

解决方案

我已经设法解决这个问题，为每个'group'使用Hibernate Sessions插入。结果是保存数据所需时间减少了大约7倍。过去大约需要2000毫秒才能保存一个'数据包'，现在需要200毫秒到300毫秒才能完成同样的任务。

只需重复一遍 - 这对 Play！有效！ Framework 1.2.3 - 我不确定是否或者如何适用于其他使用Hibernate的框架或应用程序。

  Session mySession =（Session）Pressure.em（）。getDelegate（）; 
 
 for（int i = 0; i< data.size（）; i ++）{
 initializeFromJsonAndSave（data.get（i），mySession）; 
} 
 s.flush（）; 
 s.clear（）; 
  

'initializeFromJsonAndSave'方法已更改，因此不必调用对象的 save（） 方法，调用 mySession.save（myNewObject）。

I have a data model that has a ONE TO MANY relationship between ONE entity and 11 other entities. These 12 entities together represent one data packet. The problem I am having is to do with the number of inserts that occur on the 'many' side of these relationships. Some of them can have as many as 100 individual values so to save one whole data packet in the database it requires up to 500 inserts.

I am using MySQL 5.5 with InnoDB tables. Now, from testing the database I see that it can easily do 15000 inserts per second when processing a batch insert (and even more with LOAD DATA, but that's not practical for this case).

Is there some way to bunch up these individual 500 inserts into, say - 5 inserts with 100 VALUES (for the 5 linked entities that each has 100 values) using Hibernate?

As Requested:

@OneToMany(mappedBy="beat", cascade=CascadeType.ALL)
@OrderBy("miliseconds ASC")
public List<AmbientLight> lights;

I should probably also mention one important piece of information - I am using Play! Framework 1.2.3

解决方案

I have managed to solve this problem by using Hibernate Sessions for each 'group' of inserts. The results are about a 7-fold reduction in time needed to save the data. Used to take approximately 2000ms to save one 'packet' and now it takes between 200ms and 300ms to do the same thing.

Just to repeat - this is valid for Play! Framework 1.2.3 - I am not sure whether, or how this applies to other frameworks or applications that utilize Hibernate.

    Session mySession = (Session) Pressure.em().getDelegate();

    for(int i = 0 ; i < data.size() ; i++){
        initializeFromJsonAndSave(data.get(i), mySession);
    }
    s.flush();
    s.clear();

The 'initializeFromJsonAndSave' method was changed so that, instead of calling the object's save() method, calls mySession.save(myNewObject).

这篇关于JPA / Hibernate提高批量插入性能的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！