在PHP中读取非常大的文件 [英] Reading very large files in PHP

问题描述

fopen 当我尝试在 PHP 中读取非常适中的文件时，失败。 一个6兆文件使它窒息，虽然较小的文件围绕 100k 是正常的。我读过，为了读取文件，有时需要重新编译 PHP 与 -D_FILE_OFFSET_BITS = 64 标志超过20个演出或可笑的东西，但是我不应该没有问题的6兆文件？最终，我们将要读取大约100万像素的文件，很高兴能够打开它们，然后仔细阅读它们与fgets的一行，因为我可以使用较小的文件。

fopen is failing when I try to read in a very moderately sized file in PHP. A 6 meg file makes it choke, though smaller files around 100k are just fine. i've read that it is sometimes necessary to recompile PHP with the -D_FILE_OFFSET_BITS=64 flag in order to read files over 20 gigs or something ridiculous, but shouldn't I have no problems with a 6 meg file? Eventually we'll want to read in files that are around 100 megs, and it would be nice be able to open them and then read through them line by line with fgets as I'm able to do with smaller files.

您在 PHP 中阅读和执行非常大文件的操作的技巧/解决方案是什么？

What are your tricks/solutions for reading and doing operations on very large files in PHP?

更新：这是一个简单的代码块在我的6兆文件失败的例子 - PHP似乎没有抛出错误，它只是返回false。也许我正在做一些非常愚蠢的事情？

Update: Here's an example of a simple codeblock that fails on my 6 meg file - PHP doesn't seem to throw an error, it just returns false. Maybe I'm doing something extremely dumb?

$rawfile = "mediumfile.csv";

if($file = fopen($rawfile, "r")){  
  fclose($file);
} else {
  echo "fail!";
}

另一个更新：感谢所有的帮助，它确实变成了一些东西令人难以置信的愚蠢 - 一个权限问题。当较大的文件没有时，我的小文件不可思议地具有读取权限。 Doh！

Another update: Thanks all for your help, it did turn out to be something incredibly dumb - a permissions issue. My small file inexplicably had read permissions when the larger file didn't. Doh!

推荐答案

你确定它是 fopen 您的脚本的超时设置？默认值通常在30秒左右，如果您的文件的读取时间长于该文件，则可能会跳过该文件。

Are you sure that it's fopen that's failing and not your script's timeout setting? The default is usually around 30 seconds or so, and if your file is taking longer than that to read in, it may be tripping that up.

另一件需要考虑的事项作为您的脚本的内存限制 - 将文件读入数组可能会跳过此操作，因此请检查您的错误日志以获取内存警告。

Another thing to consider may be the memory limit on your script - reading the file into an array may trip over this, so check your error log for memory warnings.

如果上述都不是您的问题，您可以查看使用 fgets 阅读该文件逐行处理，您可以随时处理。

If neither of the above are your problem, you might look into using fgets to read the file in line-by-line, processing as you go.

$handle = fopen("/tmp/uploadfile.txt", "r") or die("Couldn't get handle");
if ($handle) {
    while (!feof($handle)) {
        $buffer = fgets($handle, 4096);
        // Process buffer here..
    }
    fclose($handle);
}

修改

PHP似乎没有发生错误，它只是返回false。

PHP doesn't seem to throw an error, it just returns false.

路径到 $ rawfile 相对于脚本运行的位置是否正确？也许尝试在文件名中设置一个绝对路径。

Is the path to $rawfile correct relative to where the script is running? Perhaps try setting an absolute path here for the filename.

这篇关于在PHP中读取非常大的文件的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！