用 PHP 读取非常大的文件 [英] Reading very large files in PHP

问题描述

fopen 失败.一个 6 meg 的文件 让它窒息，尽管 100k 左右的小文件就好了.我读过有时需要使用 -D_FILE_OFFSET_BITS=64 标志重新编译 PHP 以读取超过 20 个演出或一些荒谬的文件，但我不应该6 兆文件没有问题吗?最终，我们希望读取大约 100 兆字节的文件，如果能够打开它们，然后使用 fgets 逐行读取它们，就像我能够处理较小的文件一样.

fopen is failing when I try to read in a very moderately sized file in PHP. A 6 meg file makes it choke, though smaller files around 100k are just fine. i've read that it is sometimes necessary to recompile PHP with the -D_FILE_OFFSET_BITS=64 flag in order to read files over 20 gigs or something ridiculous, but shouldn't I have no problems with a 6 meg file? Eventually we'll want to read in files that are around 100 megs, and it would be nice be able to open them and then read through them line by line with fgets as I'm able to do with smaller files.

你在PHP中读取和操作超大文件的技巧/解决方案是什么?

What are your tricks/solutions for reading and doing operations on very large files in PHP?

更新:这是一个在我的 6 meg 文件中失败的简单代码块示例 - PHP 似乎没有抛出错误，它只是返回 false.也许我在做一些非常愚蠢的事情?

Update: Here's an example of a simple codeblock that fails on my 6 meg file - PHP doesn't seem to throw an error, it just returns false. Maybe I'm doing something extremely dumb?

$rawfile = "mediumfile.csv";

if($file = fopen($rawfile, "r")){  
  fclose($file);
} else {
  echo "fail!";
}

另一个更新:感谢大家的帮助，结果证明这是一件非常愚蠢的事情 - 权限问题.当大文件没有时，我的小文件莫名其妙地具有读取权限.呸！

Another update: Thanks all for your help, it did turn out to be something incredibly dumb - a permissions issue. My small file inexplicably had read permissions when the larger file didn't. Doh!

推荐答案

您确定是 fopen 失败而不是脚本的超时设置?默认值通常约为 30 秒左右，如果您的文件读入时间超过该时间，则可能会导致文件中断.

Are you sure that it's fopen that's failing and not your script's timeout setting? The default is usually around 30 seconds or so, and if your file is taking longer than that to read in, it may be tripping that up.

要考虑的另一件事可能是脚本的内存限制 - 将文件读入数组可能会导致此问题，因此请检查错误日志以获取内存警告.

Another thing to consider may be the memory limit on your script - reading the file into an array may trip over this, so check your error log for memory warnings.

如果以上都不是您的问题，您可以考虑使用 fgets 逐行读取文件，随时处理.

If neither of the above are your problem, you might look into using fgets to read the file in line-by-line, processing as you go.

$handle = fopen("/tmp/uploadfile.txt", "r") or die("Couldn't get handle");
if ($handle) {
    while (!feof($handle)) {
        $buffer = fgets($handle, 4096);
        // Process buffer here..
    }
    fclose($handle);
}

编辑

PHP 似乎没有抛出错误，它只是返回 false.

PHP doesn't seem to throw an error, it just returns false.

$rawfile 的路径相对于脚本运行的位置是否正确?也许尝试在此处为文件名设置绝对路径.

Is the path to $rawfile correct relative to where the script is running? Perhaps try setting an absolute path here for the filename.

这篇关于用 PHP 读取非常大的文件的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！