php / symfony / doctrine内存泄漏？ [英] php/symfony/doctrine memory leak?

问题描述

我使用symfony 1.4和doctrine 1.2将对象批量插入到数据库中遇到问题。

I'm having problems with a batch insertion of objects into a database using symfony 1.4 and doctrine 1.2.

我的模型有一种称为扇区，每个都有几个Cupo类型的对象（通常从50到200000）。这些物体很小;只是一个短的标识符字符串和一个或两个整数。每当一组扇区由用户创建时，我需要自动将所有这些Cupo实例添加到数据库。万一有什么问题，我正在使用一个学说交易来回滚一切。问题是，在PHP内存不足之前，我只能创建大约2000个实例。它目前具有128MB的限制，这对于处理使用少于100个字节的对象应该是足够的。我尝试将内存限制增加到512MB，但php仍然崩溃，这并不能解决问题。我正在批量插入是否正确或有更好的方法？

My model has a certain kind of object called "Sector", each of which has several objects of type "Cupo" (usually ranging from 50 up to 200000). These objects are pretty small; just a short identifier string and one or two integers. Whenever a group of Sectors are created by the user, I need to automatically add all these instances of "Cupo" to the database. In case anything goes wrong, I'm using a doctrine transaction to roll back everything. The problem is that I can only create around 2000 instances before php runs out of memory. It currently has a 128MB limit, which should be more than enough for handling objects that use less than 100 bytes. I've tried increasing the memory limit up to 512MB, but php still crashes and that doesn't solve the problem. Am I doing the batch insertion correctly or is there a better way?

这是错误：

Fatal error: Allowed memory size of 134217728 bytes exhausted (tried to allocate 71 bytes) in /Users/yo/Sites/grifoo/lib/vendor/symfony/lib/log/sfVarLogger.class.php on line 170

这里是代码：

public function save($conn=null){

    $conn=$conn?$conn:Doctrine_Manager::connection();

    $conn->beginTransaction();


    try {
        $evento=$this->object;


        foreach($evento->getSectores() as $s){

            for($j=0;$j<$s->getCapacity();$j++){

                $cupo=new Cupo();
                $cupo->setActivo($s->getActivo());
                $cupo->setEventoId($s->getEventoId());
                $cupo->setNombre($j);
                $cupo->setSector($s);

                $cupo->save();

            }
        }

        $conn->commit();
        return;
    }
    catch (Exception $e) {
        $conn->rollback();
        throw $e;
    }

再次，该代码对于不到1000个对象可以正常工作，但任何更大的超过1500次失败。感谢您的帮助。

Once again, this code works fine for less than 1000 objects, but anything bigger than 1500 fails. Thanks for the help.

推荐答案

尝试

$cupo->save();
$cupo->free();
$cupo = null;

（但代替我的代码）而且我仍然得到内存溢出。任何其他想法，SO？

(But substituting my code) And I'm still getting memory overflows. Any other ideas, SO?

更新：

我创建了一个新的环境在我的databases.yml中，看起来像：

I created a new environment in my databases.yml, that looks like:

all:
  doctrine:
    class: sfDoctrineDatabase
    param:
      dsn: 'mysql:host=localhost;dbname=.......'
      username: .....
      password: .....
      profiler: false

profiler：false 查询日志记录，通常会保留您所做的每个查询的副本。它没有停止内存泄漏，但是我能够获得大约两倍的数据导入，因为我没有它。

The profiler: false entry disables doctrine's query logging, that normally keeps a copy of every query you make. It didn't stop the memory leakage, but I was able to get about twice as far through my data importing as I was without it.

更新2

我添加了

Doctrine_Manager::connection()->setAttribute(Doctrine_Core::ATTR_AUTO_FREE_QUERY_OBJECTS, true ); 

在运行我的查询之前，更改了

before running my queries, and changed

$cupo = null;

至

unset($cupo);

现在，我的脚本已经很开心了。我很确定这次会不会耗尽RAM。

And now my script has been churning away happily. I'm pretty sure it will finish without running out of RAM this time.

更新3

这是获奖的组合。

这篇关于php / symfony / doctrine内存泄漏？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！