而在foreach循环使用array_combine PHP内存枯竭型 [英] PHP memory exhaused while using array_combine in foreach loop

问题描述

当试图在的foreach 循环中使用 array_combine 我有一个麻烦。这将结束一个错误：

I'm having a trouble when tried to use array_combine in a foreach loop. It will end up with an error:

PHP Fatal error:  Allowed memory size of 268435456 bytes exhausted (tried to allocate 85 bytes) in

下面是我的code：

$data = array();
$csvData = $this->getData($file);
if ($columnNames) {
    $columns = array_shift($csvData);
    foreach ($csvData as $keyIndex => $rowData) {
        $data[$keyIndex] = array_combine($columns, array_values($rowData));
    }
}

return $data;

我已经使用的源文件CSV具有约〜1,000,000行。该行

The source file CSV which I've used has approx ~1,000,000 rows. This row

$csvData = $this->getData($file)

我是用whil​​e循环读取CSV并将其分配到一个数组，它的工作没有任何问题。麻烦来自 array_combine 和的foreach 循环。

你有任何想法解决这个或者只是有一个更好的解决方案？

Do you have any idea to resolve this or simply have a better solution?

下面是code读取CSV文件（使用whil​​e循环）

Here is the code to read the CSV file (using while loop)

$data = array();
if (!file_exists($file)) {
    throw new Exception('File "' . $file . '" do not exists');
}

$fh = fopen($file, 'r');
while ($rowData = fgetcsv($fh, $this->_lineLength, $this->_delimiter, $this->_enclosure)) {
    $data[] = $rowData;
}
fclose($fh);
return $data;

更新2

在code以上没有工作，如果你是一个CSV文件＆LT玩弄任何问题; = 20,000以上三万元的行。从50,000行及以上，内存将被耗尽。

UPDATED 2

The code above is working without any problem if you are playing around with a CSV file <=20,000~30,000 rows. From 50,000 rows and up, the memory will be exhausted.

推荐答案

你其实保持（或试图保持）在你的记忆整个数据集的两个不同的副本。首先，你使用的getData（）加载整个CSV日期到内存中，并且将数据复制到 $数据阵列通过循环在内存中的数据，并创建一个新的数组。

You're in fact keeping (or trying to keep) two distinct copies of the whole dataset in your memory. First you load the whole CSV date into memory using getData() and the you copy the data into the $data array by looping over the data in memory and creating a new array.

加载CSV数据仅保存一个数据集在内存中时，您应该使用基于流的阅读。如果你在PHP 5.5+（你绝对应该的方式），这是一个简单的改变你的的getData 方法看起来像：

You should use stream based reading when loading the CSV data to keep just one data set in memory. If you're on PHP 5.5+ (which you definitely should by the way) this is a simple as changing your getData method to look like that:

protected function getData($file) {
    if (!file_exists($file)) {
        throw new Exception('File "' . $file . '" do not exists');
    }

    $fh = fopen($file, 'r');
    while ($rowData = fgetcsv($fh, $this->_lineLength, $this->_delimiter, $this->_enclosure)) {
        yield $rowData;
    }
    fclose($fh);
}

这是利用所谓的的生成 这是一个PHP> = 5.5功能。您code的其余部分应继续为的getData 的内部运作的工作应该是透明的调用code（只有真理的一半）。

This makes use of a so-called generator which is a PHP >= 5.5 feature. The rest of your code should continue to work as the inner workings of getData should be transparent to the calling code (only half of the truth).

更新，以解释如何提取的列标题将现在的工作。

UPDATE to explain how extracting the column headers will work now.

$data = array();
$csvData = $this->getData($file);
if ($columnNames) { // don't know what this one does exactly
    $columns = null;
    foreach ($csvData as $keyIndex => $rowData) {
        if ($keyIndex === 0) {
            $columns = $rowData;
        } else {
            $data[$keyIndex/* -1 if you need 0-index */] = array_combine(
                $columns, 
                array_values($rowData)
            );
        }
    }
}

return $data;

这篇关于而在foreach循环使用array_combine PHP内存枯竭型的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！