Doctrine2 ORM - 持续运作需要很长时间 [英] Doctrine2 ORM - persist operations taking a long time

问题描述

我有一个使用Doctrine2 ORM的Symfony2应用程序。

我正在尝试从XML文件创建实体，然后持久化实体。典型的XML文件可能包含需要持久存储的几千条记录。 XML中的每个记录都不会直接映射到单个实体上，而是映射到一个实体，然后是一对多关系的许多一方的其他实体。

我可以从XML元素创建实体，但是在实体上运行持久，每个操作在我的机器上大约需要2秒钟。从XML文件导入数千条记录，这对我们的需求来说太慢了。

任何人都可以提供任何帮助？

解决方案

请参阅在原则文档中进行批处理。这个想法是在每个新实体上调用 persist（），但只有在 n 实体持久化。为每个实体调用 persist（） then flush（）将花费更少的时间。

例如：

  $ batchSize = 20; 
（$ i = 1; $ i< = 10000; ++ $ i）{
 $ user = new CmsUser; 
 $ user-> setStatus（'user'）; 
 $ user-> setUsername（'user'。$ i）; 
 $ user-> setName（'Mr.Smith-'。$ i）; 
 $ em-> persist（$ user）; 
 if（（$ i％$ batchSize）=== 0）{
 $ em-> flush（）; 
} 
} 
  

我删除了 clear（） 因为它会分离所有的实体。这是一个问题，如果你使用一个 foreach（）{} 与实体，因为Doctrine2将分离实体，循环将是破坏。

不使用 clear（），Doctrine2保留所有持久化实体的内存，如果它比PHP可以使用更多的内存。

如果你在Doctrine存储库之外重复循环，那么你可以调用 clear（） / code> after flush（）。

I have a Symfony2 application that uses Doctrine2 ORM.

I am trying to create entities from an XML file, and then persist the entities. A typical XML file might contain a few thousand records that need to be persisted. Each record in the XML does not map directly onto a single entity, but onto an entity and then some other entities that are on the "many" side of a one-to-many relationship.

I can create the entity from the XML elements, but it comes to running "persist" on the entity, each operation takes around 2 seconds on my machine. With several thousand records being imported from the XML file, this is too slow for our needs.

Could anyone offer any help?

解决方案

See Batch Processing in Doctrine documentation. The idea is to call persist() on each new entity but use flush() only after a group of n entities are persisted. It will take less time than calling persist() then flush() for every entity.

For example:

$batchSize = 20;
for ($i = 1; $i <= 10000; ++$i) {
    $user = new CmsUser;
    $user->setStatus('user');
    $user->setUsername('user' . $i);
    $user->setName('Mr.Smith-' . $i);
    $em->persist($user);
    if (($i % $batchSize) === 0) {
        $em->flush();
    }
}

I removed clear() since it will detach all the entities. It's a problem if for example you used a foreach() {} with entities, since Doctrine2 would detach the entities, the loop would be broken.

Without using clear(), Doctrine2 keep in memory all the persisted entities, it can cause an error if it takes more memory than PHP can use.

If you are iterating the loop on something else than Doctrine repositories, then you can call clear() after flush().

这篇关于Doctrine2 ORM - 持续运作需要很长时间的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！