好主意运行一个PHP文件几个小时作为cronjob？ [英] Good idea to run a PHP file for a few hours as cronjob?

问题描述

我想每天晚上运行一个PHP脚本作为cronjob。 PHP脚本将导入一个包含大约145.000个产品的XML文件。每个产品都包含一个指向图像的链接，它将被下载并保存在服务器上。我可以想象，这可能会导致一些超载。所以我的问题是：它是一个更好的主意，拆分PHP文件？如果是，什么会是更好的解决方案？更多的cronjobs，几分钟之间相互之间暂停？使用exec运行另一个PHP文件（猜测，因为我不能想象这会有很大的区别），或者其他...？

解决方案

/ div>

这取决于你是如何编写它是否泄漏打开的文件或数据库连接。它还取决于您使用的是哪个版本的PHP。在php 5.3中，有很多工作来处理垃圾回收：

http://www.php.net/manual/en/features.gc.performance-considerations.php

如果操作是事务性的并不重要，例如全部或没有（例如，如果它中途失败），那么我会试图在大块中解决这个问题，其中每次运行脚本处理下一个x项目，其中x可以是一个变量，取决于它需要多长时间。所以你需要做的是继续重复脚本，直到没有做任何事情。

为了做到这一点，我建议使用一个工具叫Fat控制器：

http://fat-controller.sourceforge.net

它可以继续重复脚本，然后一旦完成就停止。你可以告诉Fat控制器有更多的事情，或者使用退出状态从php脚本完成一切。 Fat控制器网站上有一些用例，例如： http：/ /fat-controller.sourceforge.net/use-cases.html#generating-newsletters

您还可以使用Fat控制器并行运行进程速度的东西，只是要小心你不要并行运行太多，慢下来。如果你正在写一个数据库，那么最终你将受到硬盘的限制，除非你有某种想法，这意味着你的最佳并发将是1。

最后一个问题是如何触发这个 - 你最好是从CRON触发Fat控制器。

有很多关于Fat控制器的文档和示例网站，但如果你需要任何具体的指导，我会很乐意提供帮助。

I would like to run a PHP script as a cronjob every night. The PHP script will import a XML file with about 145.000 products. Each product contains a link to an image which will be downloaded and saved on the server as well. I can imagine that this may cause some overload. So my question is: is it a better idea to split the PHP file? And if so, what would be a better solution? More cronjobs, with several minutes pause between each other? Run another PHP file using exec (guess not, cause I can't imagine that would make much of a difference), or someting else...? Or just use one script to import all products at once?

Thanks in advance.

解决方案

It depends a lot on how you've written it in terms of whether it doesn't leak open files or database connections. It also depends on which version of php you're using. In php 5.3 there was a lot done to address garbage collection:

http://www.php.net/manual/en/features.gc.performance-considerations.php

If it's not important that the operation is transactional, i.e all or nothing (for example, if it fails half way through) then I would be tempted to tackle this in chunks where each run of the script processed the next x items, where x can be a variable depending on how long it takes. So what you'll need to do then is keep on repeating the script until nothing is done.

To do this, I'd recommend using a tool called the Fat Controller:

http://fat-controller.sourceforge.net

It can keep on repeating the script and then stop once everything is done. You can tell the Fat Controller that there's more to do, or that everything is done using exit statuses from the php script. There are some use cases on the Fat Controller website, for example: http://fat-controller.sourceforge.net/use-cases.html#generating-newsletters

You can also use the Fat Controller to run processes in parallel to speed things up, just be careful you don't run too many in parallel and slow things down. If you're writing to a database, then ultimately you'll be limited by the hard disc, which unless you have something fancy will mean your optimum concurrency will be 1.

The final question would be how to trigger this - and you're probably best off triggering the Fat Controller from CRON.

There's plenty of documentation and examples on the Fat Controller website, but if you need any specific guidance then I'd be happy to help.

这篇关于好主意运行一个PHP文件几个小时作为cronjob？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！