如何解决这个教条内存问题 [英] How can I resolve this doctrine memory issue

问题描述

以下代码遍历具有〜1,000,000行的表。我被建议使用Doctrine的iterateResult（）方法来避免内存问题（ docs ）：

The following code iterates over a table which has ~1,000,000 rows. I was advised to use Doctrine's iterateResult() method to avoid memory issues (docs) :

$batchSize = 1000;
    $i = 0;
    $q = $em->createQuery('SELECT i from MyBundle:Items i WHERE i.imgSize IS NULL');

    $results = array();

    $iterableResult = $q->iterate();
    while (($row = $iterableResult->next()) !== false) {

        $results[]=$row[0];
        $em->detach($row[0]);

        if (($i % $batchSize) === 0) {

            $em->clear(); // allegedly detaches all objects from Doctrine!
            $this->chunkedWhereCallback($results);

        }
        ++$i;
    }

上述代码超过允许的内存，即使我将批量大小设置为1。如果我更改查询以读取 SELECT i.id，i.img FROM MyBundle：Items i WHERE i.imgSize is NULL 然后内存错误不存在，但是我的$ row数组没有[0]键，例如：

The above code exceeds allowed memory even if I set the batch size to 1. If I alter the query to read SELECT i.id, i.img FROM MyBundle:Items i WHERE i.imgSize IS NULL then the memory errors are absent but then my $row array does not have a [0] key e.g:

array(1) {
  [5926] =>
  array(2) {
    'id' =>
    int(12794)
    'img' =>
    string(57) "http://some/url/1203/104962420.jpg"
  }
}

有人可以告诉我我在这里失踪了吗？ Laravel的口才处理这个操作很好，但我想将代码移植到Doctrine / Symfony2。我已经尝试将php内存限制设置为1024M，这应该足够多，超出了我在服务器上可用的数量。

Can someone tell me what I'm missing here? Laravel's Eloquent handled this operation just fine but I wanted to port the code to Doctrine / Symfony2. I have tried setting the php memory limit to 1024M which should be more than enough and is more than I have available on the server.

Memory error is Fatal error: Allowed memory size of 1073741824 bytes exhausted (tried to allocate 32 bytes) in /Users/mre/Sites/ site/vendor/doctrine/dbal/lib/Doctrine/DBAL/Connection.php on line 699

推荐答案

我也遇到同样的问题，我不完全确定这是Doctrine的问题，似乎是PHP本身的一个问题。

I also encountered the same problem and I'm not totally sure it is a problem with Doctrine, it seems like a issue with PHP itself.

无论如何，我发现解决方案是强制PHP的垃圾回收发生如下： p>

Anyway the solution I found was to force PHP's garbage collection to occur like so:

$batchSize = 1000;
$i = 0;
$q = $em->createQuery('SELECT i from MyBundle:Items i WHERE i.imgSize IS NULL');

$results = array();

$iterableResult = $q->iterate();
while (($row = $iterableResult->next()) !== false) {

    $results[]=$row[0];
    $em->detach($row[0]);

    if (($i % $batchSize) === 0) {

        $em->clear(); // allegedly detaches all objects from Doctrine!
        // Force PHP's GC
        gc_collect_cycles();
        $this->chunkedWhereCallback($results);

    }
    ++$i;
}

调用 gc_collect_cycles 应该释放内存并防止可用内存耗尽。

The call to gc_collect_cycles should free the memory and prevent exhaustion of the available memory.

这篇关于如何解决这个教条内存问题的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！