在大事务中安全地清理Hibernate会话 [英] Safely clearing Hibernate session in the middle of large transaction

问题描述

我正在使用Spring + Hibernate进行一项操作，该操作需要创建和更新几十万个项目。像这样：

  {
 ... 
 Foo foo = fooDAO.get（... ）; 
 for（int i = 0; i <500000; i ++）{
 Bar bar = barDAO.load（i）; 
 if（bar.needsModification（）& foo.foo（））{
 bar.setWhatever（new whatever）; 
 barDAO.update（bar）; 
 // commit here 
 Baz baz = new Baz（）; 
 bazDAO.create（baz）; 
 // if（i％100 == 0），clear 
} 
} 
} 
  

为了保护自己免受中间变更的损失，我立即在 barDAO.update（bar）：

  HibernateTransactionManager transactionManager = ...; //由Spring注入
 DefaultTransactionDefinition def = new DefaultTransactionDefinition（）; 
 def.setPropagationBehavior（TransactionDefinition.PROPAGATION_REQUIRED）; 
 TransactionStatus transactionStatus = transactionManager.getTransaction（def）; 
 transactionManager.commit（transactionStatus）; 
  

在这一点上，我必须说整个过程运行在一个事务中，包装到 org.springframework.orm.hibernate3.support.ExtendedOpenSessionInViewFilter （是的，这是一个web应用程序）。

例外：在数千次更新/提交之后，整个过程变得非常缓慢，很可能是由于内存由于Spring / Hibernate保存的不断增加的对象而变得臃肿。

在仅限Hibernate的环境中，通过调用 org.hibernate.Session＃clear（）很容易解决这个问题。

现在，问题：

• 什么时候是明确（）？它有很大的性能成本吗？


• 为什么不是像 bar 或 baz 自动发布/ GCd？在提交后将它们保留在会话中的重点是什么（在迭代的下一个循环中，它们无法访问）？我没有做内存转储来证明这一点，但我的良好感觉是，他们仍然在那里，直到完全退出。如果这个答案是Hibernate缓存，那么为什么缓存刷新可用内存不足？


• 安全/建议调用 org.hibernate.Session＃clear（）直接（考虑到整个Spring上下文，例如延迟加载等）？是否有任何可用的Spring包装/对应物实现相同？


• 如果对上述问题的回答为真，对象 foo ，假设 clear（）在循环中被调用？如果 foo.foo（）是一个延迟加载方法？ 感谢您的回答。

  解决方案

  
  

  什么时候清除（）？它是否有很高的性能成本？
  

  在刷新更改后，按照固定的时间间隔，理想情况下与JDBC批处理大小相同。该文档在有关批处理的章节中描述常见习惯用法：

  
  

  13.1。批量插入

  
  
  

  定期创建新对象
  flush（）然后清除（）会话
  以控制大小为
  的一级缓存。

    Session session = sessionFactory.openSession（）; 
   Transaction tx = session.beginTransaction（）; 
   
  （int i = 0; i <100000; i ++）{
   Customer customer = new Customer（.....）; 
   session.save（customer）; 
   if（i％20 == 0）{// 20，与JDBC批量大小相同
   //刷新一批插入并释放内存：
   session.flush（）; 
   session.clear（）; 
  } 
  } 
   
   tx.commit（）; 
   session.close（）; 
    

  
  

  这不应该有一个性能 cost ，au contraire：
  
  

  
  
  • 它允许保持对象的数量跟踪肮脏低（因此冲洗应该很快），
  
  
  • 它应该允许回收记忆。
  
  
  
  
  

  
  

  为什么选择aren像bar或baz这样的对象会自动释放/ GCd？在提交后将它们保留在会话中的点是什么（在迭代的下一个循环中它们无法到达）？
  

  如果您不想保留实体，那么您需要明确地显示会话 clear（）这就是它的工作原理（人们可能希望在没有放弃实体）。
  
  

  但是从我所看到的情况来看，bar和baz实例应该在明确之后成为GC的候选人。分析内存转储以查看到底发生了什么会很有趣。
  
  

  
  

  安全/建议调用org.hibernate.Session直接使用#clear（）
  

  只要您 flush（）改变不松散他们（除非这是你想要的），我没有看到任何问题（你当前的代码将失去一个创建每100循环，但也许它只是一些伪代码）。
  $ b

  
  

  如果对上述问题的回答是正确的，假设在循环内部调用clear（），对象foo会发生什么？如果foo.foo（）是一个延迟加载方法？

  
  

  调用 clear（） 清除所有已载入的实例从 Session ，使他们分离的实体。如果后续调用需要实体附加，则会失败。
  

  I am using Spring+Hibernate for an operation which requires creating and updating literally hundreds of thousands of items. Something like this:

  {
     ...
     Foo foo = fooDAO.get(...);
     for (int i=0; i<500000; i++) {
        Bar bar = barDAO.load(i);
        if (bar.needsModification() && foo.foo()) {
           bar.setWhatever("new whatever");
           barDAO.update(bar);
           // commit here
           Baz baz = new Baz();
           bazDAO.create(baz);
           // if (i % 100 == 0), clear
        }
     }
  }
  

  To protect myself against losing changes in the middle, I commit the changes immediately after barDAO.update(bar):

  HibernateTransactionManager transactionManager = ...; // injected by Spring
  DefaultTransactionDefinition def = new DefaultTransactionDefinition();
  def.setPropagationBehavior(TransactionDefinition.PROPAGATION_REQUIRED);
  TransactionStatus transactionStatus = transactionManager.getTransaction(def);
  transactionManager.commit(transactionStatus);
  

  At this point I have to say that entire process is running in a transaction wrapped into org.springframework.orm.hibernate3.support.ExtendedOpenSessionInViewFilter (yes, this is a webapp).

  This all works fine with one exception: after few thousand of updates/commits, entire process gets really slow, most likely due to memory being bloated by ever-increasing amount of objects kept by Spring/Hibernate.

  In Hibernate-only environment this would be easily solvable by calling org.hibernate.Session#clear().

  Now, the questions:

  • When is it a good time to clear()? Does it have big performance cost?
  • Why aren't objects like bar or baz released/GCd automatically? What's the point of keeping them in the session after the commit (in the next loop of iteration they're not reachable anyway)? I haven't done memory dump to prove this but my good feeling is that they're still there until completely exited. If the answer to this is "Hibernate cache", then why isn't the cache flushed upon the available memory going low?
  • is it safe/recommended to call org.hibernate.Session#clear() directly (having in mind entire Spring context, things like lazy loading, etc.)? Are there any usable Spring wrappers/counterparts for achieving the same?
  • If answer to the above question is true, what will happen with object foo, assuming clear() is called inside the loop? What if foo.foo() is a lazy-load method?

  Thank you for the answers.

  解决方案

  When is it a good time to clear()? Does it have big performance cost?

  At regular intervals, ideally the same as the JDBC batch size, after having flushed the changes. The documentation describes common idioms in the chapter about Batch processing:

  13.1. Batch inserts

  When making new objects persistent flush() and then clear() the session regularly in order to control the size of the first-level cache.

  Session session = sessionFactory.openSession();
  Transaction tx = session.beginTransaction();
  
  for ( int i=0; i<100000; i++ ) {
      Customer customer = new Customer(.....);
      session.save(customer);
      if ( i % 20 == 0 ) { //20, same as the JDBC batch size
          //flush a batch of inserts and release memory:
          session.flush();
          session.clear();
      }
  }
  
  tx.commit();
  session.close();
  

  And this shouldn't have a performance cost, au contraire:

  • it allows to keep the number of objects to track for dirtiness low (so flushing should be fast),
  • it should allow to reclaim memory.

  Why aren't objects like bar or baz released/GCd automatically? What's the point of keeping them in the session after the commit (in the next loop of iteration they're not reachable anyway)?

  You need to clear() the session explicitly if you don't want to keep entities tracked, that's all, that's how it works (one might want to commit a transaction without "loosing" the entities).

  But from what I can see, bar and baz instances should become candidate to GC after the clear. It would be interesting to analyze a memory dump to see what is happening exactly.

  is it safe/recommended to call org.hibernate.Session#clear() directly

  As long as you flush() the pending changes to not loose them (unless this is what you want), I don't see any problem with that (your current code will loose a create every 100 loop but maybe it's just some pseudo code).

  If answer to the above question is true, what will happen with object foo, assuming clear() is called inside the loop? What if foo.foo() is a lazy-load method?

  Calling clear() evicts all loaded instances from the Session, making them detached entities. If a subsequent invocation requires an entity to be "attached", it will fail.

  这篇关于在大事务中安全地清理Hibernate会话的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！