PHP应用程序内存不足 [英] PHP application running out of Memory

问题描述

我正在为爬虫编写一组类，它爬虫一个起始页面，根据参数拉三个链接(使用Simple Html Dom Parser找到，允许使用像选择器一样的jquery)，对那些页面进行爬虫，然后转到第2页，选择接下来的3页.当前的最大页面数是57次.

I am writing a set of classes for a crawler, it crawls a start page, pulls three links based on parameters (found using Simple Html Dom Parser allowing use of jquery like selectors), crawls those pages, then goes to page 2, picks the next 3 pages. Current max pages is 57 times.

不用说我正在得到:

允许的50331648字节的内存大小用尽错误消息.

Allowed memory size of 50331648 bytes exhausted error message.

有什么办法可以避免内存不足.

Is there any way I can avoid running out of memmory.

为了让您知道，在拉入第一页的内容之后，我运行了go()函数，该函数连续拉入页面，直到达到$ this-> maxpages为止.我想我可以在实例化类时运行循环，但这会有所帮助.

To let you know, after pulling in the contents of the first page, I run a go() function, which continuosly pulls in the pages until $this->maxpages is reached. I suppose I could run the loop when instantiating the classes, but would this help.

推荐答案

您可以调整内存限制:

ini_set('memory_limit', '128M');

但是我会尝试使脚本使用更少的内存.确保释放数据/对不再需要的任何内容的引用.

But I'd try to make the script use less memory. Make sure you are freeing data / references to anything that no longer needs to exist.

memory_get_usage()在调试内存使用累积的地方很有用.

memory_get_usage() can be useful in debugging where the memory usage is accumulating.

此外，如果您不使用PHP 5.3，则可以考虑进行升级，因为它的垃圾收集器更好.

Also, if you aren't using PHP 5.3, you may consider upgrading since its garbage collector is better.

这篇关于PHP应用程序内存不足的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！