php多线程问题 [英] php multithreading problem

问题描述

我编写一个php cron作业，使用curl读取数千个订阅源/网页，并将内容存储在数据库中。我如何限制线程数，比方说，6？即，即使我需要扫描数千个订阅源/网页，我只需要6个curl线程在任何时间活动，使我的服务器和网络不会陷入困境。我可以很容易地在Java中使用Object的等待，通知，notifyall方法。我应该创建自己的信号量还是php提供任何内置函数？

解决方案

首先，PHP没有线程，但它有流程控制：
http://php.net /manual/en/book.pcntl.php

我已经围绕这些函数构建了一个类来帮助我的多进程需求。

我也有类似的情况。我保留从cron和他们的状态开始的进程的日志。

$ b

$ b

在我的项目中，我记录所有关键的更改到数据库。如果变化满足动作标准，则可以采取动作。所以我在做什么不同于你。但是，有一些相似之处。

当我分叉一个新的进程，我在一个DB表中输入它的pid。下一次cron作业开始时，它的部分功能是检查进程是否正确完成，然后在该DB表中将操作标记为已完成。

您不会给出关于您的项目的许多细节。因此，我只会抛出一个建议：

• 数据库表保存您要下载的资源的URL。


• 另一个表保存正在运行的进程的pid。


• 每小时运行的cron作业将遍历该表并下载资源并将其存储在DB 。但是，首先它检查pid表中的完整/死/运行进程，并相应地采取行动。




根据您的项目大小，这可能看起来像过度杀死。然而，我已经想了很长时间，我想跟踪所有这些分支的过程。分叉可能是有风险的业务，并且可能导致系统资源超载 - 从经验演讲）





我也有兴趣听到其他技术。




I am writing a php cron job that reads thousands of feeds / web pages using curl and stores the content in a database. How do I restrict the number of threads to, lets say, 6? i.e., even though I need to scan thousands of feeds / web pages, I want only 6 curl threads active at any time so that my server and network don't get bogged down. I could do it easily in Java using wait, notify, notifyall methods of Object. Should I build my own semaphore or does php provide any built-in functions?

解决方案

First of all, PHP doesn't have threads, but it does have process control: http://php.net/manual/en/book.pcntl.php

I've built a class around these functions to help with my multi-process requirements.

I'm in a similar situation. I'm keeping a log of the processes that get started from cron and their state. I'm checking on them from a related cron job.

EDIT (more details):

In my project I log all the key changes to the database. Actions may then be taken if the changes meet the actions criterion. So what I'm doing is different to you. However, there are some similarities.

When I fork a new process, I enter it's pid in a DB table. Then next time the cron job kicks in, part of what it does is check to see if the processes have completed properly, and then mark the action as completed in that DB table.

You don't give many details about your project. So I will just throw out a suggestion:

• A DB table holds the URLs of the resources you want to download.
• Another table holds the pids of the running processes.
• A cron job that is run every hour will go through the table and download the resource and store it in a DB. However, first it checks the pid table for complete/dead/running processes and acts accordingly. Here you can limit your processes to 6.

Depending on the size of your project, this may seem like over kill. However, I've thought about it for a long long time, and I want to keep track of all those forked processes. Forking can be risky business, and can lead to system resource overload - speaking from experience ;)

I'd be interested to hear other techniques as well.

这篇关于php多线程问题的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！