如何使用cURL与PHP同时打开多个URL？ [英] How can I use cURL to open multiple URLs simultaneously with PHP?
问题描述
这是我目前的程式码：
$ SQL = mysql_query（SELECT url FROM urls）或die（mysql_error ））; //查询urls表
while（$ resultSet = mysql_fetch_array（$ SQL））{//将所有url放入一个变量
//现在为某些cURL运行它。
$ ch = curl_init（$ resultSet ['url']）; //加载urls
curl_setopt（$ ch，CURLOPT_TIMEOUT，2）; //无需等待它加载。执行它，去。
curl_exec（$ ch）; // Execute
curl_close（$ ch）; //关闭它
} // While循环
。相对来说，我的意思是这是我第一次使用cURL。目前它加载一个两秒，然后加载下一个2秒，然后下一个。然而，我想让它加载所有的他们在同一时间。我相信它的可能，我只是不确定如何。
您将每个cURL句柄设置在同一个方法，然后将它们添加到 curl_multi _ 句柄。要查看的函数是 curl_multi _ * 函数在此处记录。根据我的经验，虽然，一次尝试加载太多的网址有问题（虽然我现在找不到我的笔记），所以最后一次我使用 curl_mutli _ ： curl_multi _ ：
编辑代码的简化版本：
// - 创建所有单独的cURL句柄并设置它们options
$ curl_handles = array（）;
foreach（$ urls as $ url）{
$ curl_handles [$ url] = curl_init（）;
curl_setopt（$ curl_handles [$ url]，CURLOPT_URL，$ url）;
//设置其他curl选项
}
// - 开始执行cURL句柄并运行它们
$ curl_multi_handle = curl_multi_init（）;
$ i = 0; // count我们在列表中，所以我们可以把运行分解成更小的块
$ block = array（）; //为每个我们同时运行的组累积curl_handles
foreach（$ curl_handles as $ a_curl_handle）{
$ i ++; //增加位置计数器
//将句柄添加到curl_multi_handle和跟踪block
curl_multi_add_handle（$ curl_multi_handle，$ a_curl_handle）;
$ block [] = $ a_curl_handle;
// - 检查我们是否有一个完全块运行，或者如果我们在结束处理列表
if（（$ i％ BLOCK_SIZE == 0）或（$ i == count（$ curl_handles）））{
// - 运行块
$ running = NULL;
do {
//跟踪上一个循环的句柄数仍然在运行，所以我们可以判断它是否改变
$ running_before = $ running;
//运行该块或检查正在运行的块，并获取仍在$ running中运行的站点数
curl_multi_exec（$ curl_multi_handle，$ running）;
//如果仍在运行的站点数量已更改，请打印一条包含仍在运行的站点数量的消息。
if（$ running！= $ running_before）{
echo（正在等待$正在运行的网站完成... \\\
）;
}
} while（$ running> 0）;
// - 一旦数字仍然运行0，curl_multi_完成，所以检查结果
foreach（$ block as $ handle）{
// HTTP响应代码
$ code = curl_getinfo（$ handle，CURLINFO_HTTP_CODE）;
// cURL错误号
$ curl_errno = curl_errno（$ handle）;
// cURL错误消息
$ curl_error = curl_error（$ handle）;
//如果出现错误，输出
if（$ curl_error）{
echo（*** cURL error：（$ curl_errno）$ curl_error\\\
） ;
}
//从curl_multi_handle中删除（used）句柄
curl_multi_remove_handle（$ curl_multi_handle，$ handle）;
}
//将块重置为空，因为我们运行了它的curl_handles
$ block = array（）;
}
}
//完成后关闭curl_multi_handle
curl_multi_close（$ curl_multi_handle）;
由于您不需要从网址返回任何内容，因此您可能不需要但是这是我如何将请求分成 BLOCK_SIZE 的块，等待每个块在继续之前运行，并从cURL捕获错误。
Here is my current code:
$SQL = mysql_query("SELECT url FROM urls") or die(mysql_error()); //Query the urls table
while($resultSet = mysql_fetch_array($SQL)){ //Put all the urls into one variable
// Now for some cURL to run it.
$ch = curl_init($resultSet['url']); //load the urls
curl_setopt($ch, CURLOPT_TIMEOUT, 2); //No need to wait for it to load. Execute it and go.
curl_exec($ch); //Execute
curl_close($ch); //Close it off
} //While loop
I'm relatively new to cURL. By relatively new, I mean this is my first time using cURL. Currently it loads one for two seconds, then loads the next one for 2 seconds, then the next. however, I want to make it load ALL of them at the same time. I'm sure its possible, I'm just unsure as to how. If someone could point me in the right direction, I'd appreciate it.
You set up each cURL handle in the same way, then add them to a curl_multi_ handle. The functions to look at are the curl_multi_* functions documented here. In my experience, though, there were issues with trying to load too many URLs at once (though I can't find my notes on it at the moment), so the last time I used curl_mutli_, I set it up to do batches of 5 URLs at a time.
edit: Here is a reduced version of the code I have using curl_multi_:
edit: Slightly rewritten and lots of added comments, which hopefully will help.
// -- create all the individual cURL handles and set their options
$curl_handles = array();
foreach ($urls as $url) {
$curl_handles[$url] = curl_init();
curl_setopt($curl_handles[$url], CURLOPT_URL, $url);
// set other curl options here
}
// -- start going through the cURL handles and running them
$curl_multi_handle = curl_multi_init();
$i = 0; // count where we are in the list so we can break up the runs into smaller blocks
$block = array(); // to accumulate the curl_handles for each group we'll run simultaneously
foreach ($curl_handles as $a_curl_handle) {
$i++; // increment the position-counter
// add the handle to the curl_multi_handle and to our tracking "block"
curl_multi_add_handle($curl_multi_handle, $a_curl_handle);
$block[] = $a_curl_handle;
// -- check to see if we've got a "full block" to run or if we're at the end of out list of handles
if (($i % BLOCK_SIZE == 0) or ($i == count($curl_handles))) {
// -- run the block
$running = NULL;
do {
// track the previous loop's number of handles still running so we can tell if it changes
$running_before = $running;
// run the block or check on the running block and get the number of sites still running in $running
curl_multi_exec($curl_multi_handle, $running);
// if the number of sites still running changed, print out a message with the number of sites that are still running.
if ($running != $running_before) {
echo("Waiting for $running sites to finish...\n");
}
} while ($running > 0);
// -- once the number still running is 0, curl_multi_ is done, so check the results
foreach ($block as $handle) {
// HTTP response code
$code = curl_getinfo($handle, CURLINFO_HTTP_CODE);
// cURL error number
$curl_errno = curl_errno($handle);
// cURL error message
$curl_error = curl_error($handle);
// output if there was an error
if ($curl_error) {
echo(" *** cURL error: ($curl_errno) $curl_error\n");
}
// remove the (used) handle from the curl_multi_handle
curl_multi_remove_handle($curl_multi_handle, $handle);
}
// reset the block to empty, since we've run its curl_handles
$block = array();
}
}
// close the curl_multi_handle once we're done
curl_multi_close($curl_multi_handle);
Given that you don't need anything back from the URLs, you probably don't need a lot of what's there, but this is how I chunked the requests into blocks of BLOCK_SIZE, waited for each block to run before moving on, and caught errors from cURL.
这篇关于如何使用cURL与PHP同时打开多个URL？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
