并行cURL请求与WRITEFUNCTION回调 [英] Parallel cURL Request with WRITEFUNCTION Callback

问题描述

我想根据这些帖子中的建议限制我的cURL响应：检索部分网页 a>和 PHP CURLOPT_WRITEFUNCTION似乎无法正常工作 。这个想法是将响应限制在回调函数中指定的4000个字符。

I'm trying to limit my cURL responses as suggested in these posts:Retrieve partial web page and PHP CURLOPT_WRITEFUNCTION doesn't appear to be working. The idea is to limit the response to 4000 characters as specified in the callback function.

我写了以下函数，但我已经知道这没有意义，因为回调函数定义中的参数不像在函数调用中那样在循环内变化。到实际调用函数时，$ key的值是固定的，所以我对这个索引的引用不会改变。

I wrote the following function, but I already know that it doesn't make sense, because a parameter in the callback function definition doesn't vary within a loop as it would within a function call. By the time the functions are actually called, the value for $key is fixed, so my references to that index won't vary.

看来我需要一个新的闭包函数，每个循环都需要引用自己的$ full_length变量。但是，我没有看到这是可能的。为了做到这一点，似乎我不得不以某种方式引用closure对象，以指定特定的$ full_length变量。

It seems that I need a new closure function for each of the loops, and each one needs to reference its own $full_length variable. However, I don't see how that's possible. In order to do that, it seems I would have to somehow make a reference the closure object in order to specify the specific $full_length variable.

任何帮助将不胜感激。感谢。

Any help would be appreciated. Thanks.

function get_headers($urls){
    $curly = array();
    $result = array();
    $mh = curl_multi_init();
    $obj = $this;
    foreach ($urls as $key => $url) {
        $this->full_length[$key] = 0;
        $callback = function ($ch, $string) use ($obj, $key){
                    $length = strlen($string);
                    $obj->full_length[$key] += $length;
                    if($obj->full_length[$key] >= 4000){
                        return -1;
                    }
                    return $length;
                };
        $curly[$key] = curl_init
        curl_setopt($curly[$key], CURLOPT_URL,            $url);
        curl_setopt($curly[$key], CURLOPT_HEADER,         0);
        curl_setopt($curly[$key], CURLOPT_WRITEFUNCTION, $callback);
        curl_setopt($curly[$key], CURLOPT_RETURNTRANSFER, 1);
        curl_multi_add_handle($mh, $curly[$key]);
    }
    $running = null;
    do {
        curl_multi_exec($mh, $running);
    } while($running > 0);
    foreach($curly as $key => $cnt) {
        $content = curl_multi_getcontent($cnt);
        curl_multi_remove_handle($mh, $cnt);
        if (strlen($content) > 0){
            $result[$key] = $content;
        } else {
            curl_multi_close($mh);
            return FALSE;
        }
    }
    curl_multi_close($mh);
    return $result;
}

编辑：

我发现一个帖子，确实是我想做的，但它是在javascript：
在for循环内部闭包 - 使用循环变量作为参数的回调。我写了以下函数尝试在PHP中做同样的事情：

I found a post that does exactly what I'm trying to do, but it's in javascript: closure inside a for loop - callback with loop variable as parameter . I wrote the following function to try to do the same thing in PHP:

function get_write_function($key){
    $this->full_length[$key] = 0;
    $obj = $this;
    $funky = function ($ch, $str) use ($obj, $key){
        $length = strlen($str);
        $obj->full_length[$key] += $length;
        if($obj->full_length[$key] >= 4000){
            return -1;
        }
        return $length;
    };
    return $funky;
}

代码运行没有错误，但它仍然没有做我想要的。关闭我的cURL句柄后，我转储了$ full_length数组，它只显示：

The code ran without errors, but it still didn't do what I wanted. After closing my cURL handles I dumped the $full_length array, and it only showed:

array([0] => 0, [1] => 0)

这表示它们是由get_write_function

That indicates that they were initialized by the get_write_function (since I didn't initialize anything in the class declaration), but that the values were never updated afterwards.

推荐答案

我终于得到了想法出来。最大的问题是，cURL忽略WRITEFUNCTION，直到我把它作为最后一个选项指定，如我在这里： cURL写功能未被调用。我实际上不需要返回转移，因为我把输出写到一个类变量。这是必要的，因为当回调返回-1时，什么都不返回。以下代码非常实用：

I finally got it figured out. The biggest problem was the fact that cURL was ignoring the WRITEFUNCTION until I placed it as the very last option specified, as I posted here: cURL WRITEFUNCTION not Being Called. I actually didn't need the return transfer, since I wrote the output to a class variable. That was necessary because when the callback returns -1, nothing gets returned. The following code works great:

var $full_length = array();
var $result = array();

function get_headers($urls){
    $curly = array();
    $mh = curl_multi_init();
    foreach ($urls as $key => $url) {
        $callback = $this->get_write_function($key);
        $curly[$key] = curl_init
        curl_setopt($curly[$key], CURLOPT_URL,            $url);
        curl_setopt($curly[$key], CURLOPT_HEADER,         0);
        curl_setopt($curly[$key], CURLOPT_WRITEFUNCTION, $callback);
        curl_multi_add_handle($mh, $curly[$key]);
    }
    $running = null;
    do {
        curl_multi_exec($mh, $running);
    } while($running > 0);
    foreach($curly as $key => $cnt) {
        curl_multi_remove_handle($mh, $cnt);
    }
    curl_multi_close($mh);
    return $this->result;
}

function get_write_function($key){
    $this->full_length[$key] = 0;
    $this->result[$key] = '';
    $obj = $this;
    $funky = function ($ch, $str) use ($obj, $key){
        $obj->result[$key] .= $str;
        $length = strlen($str);
        $obj->full_length[$key] += $length;
        if($obj->full_length[$key] >= 4000){
            return -1;
        }
        return $length;
    };
    return $funky;
}

这篇关于并行cURL请求与WRITEFUNCTION回调的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！