什么是当前最舒适可靠的跨平台Perl模块来做并行下载？ [英] What is currently the most comfortable and reliable cross-platform Perl module to do parallel downloads?

问题描述

我将不得不通过简单的POST下载一些url并获得XML来下载一些数据集。我可以通过一次多个请求来加快速度，但这里有一个钩子：

I'm going to have to download a number of datasets via simply POSTing at an url and getting XML in return. I will be able to speed this up by doing more than one request at a time, but here's the hook:

它需要在Windows和Linux上运行，所以线程和分叉都是。 （因为这是纯粹的IO绑定，我不认为他们也是需要的。）

It will need to run on both Windows and Linux, so threads and forks are both out. (Since this is purely IO-bound i don't think they're needed either.)

此外，我的同事不是在非常高的perl理解水平，但需要掌握如何使用它（不一定是发生了什么，用法是好的）。因此，如果它的 API有点简单，我会很高兴。

Additionally my coworkers aren't on a very high level of perl understanding, but need to be able to grasp how to use it (not necessarily what's going on, usage is fine). As such i'd be happy if its API was somewhat simple.

现在我正在看 IO :: Lambda 。

任何其他建议？

后遗症：根据draegtun的建议，我已经把这一点抛在脑后，完成了这个工作： https://gist.github.com/661386 您可能会在CPAN上看到它。

Post-Mortem: Based on draegtun's suggestion i've now thrown together this, which does the job perfectly: https://gist.github.com/661386 You might see it on CPAN soonish.

推荐答案

看看 AnyEvent :: HTTP 。根据 CPAN测试人员平台矩阵，它会编译&

Have a look at AnyEvent::HTTP. According to the CPAN testers platform matrix it does compile & work on Windows.

下面是一个简单的异步POSTing示例（ http_post ）。

Below is a straightforward example of async POSTing (http_post).

use 5.012;
use warnings;
use AnyEvent::HTTP;

my $cv = AnyEvent->condvar;

my @urls = (
    [google => 'http://google.com', 'some body'],
    [yahoo  => 'http://yahoo.com' , 'any body' ],
);

for my $site (@urls) {
    my ($name, $url, $body) = @$site;
    $cv->begin; 
    http_post $url, $body => sub {
        my $xml = shift;
        do_something_with_this( $name, $xml );
        $cv->end;
    }
}

# wait till all finished
$cv->recv;
say "Finished";

sub do_something_with_this { say @_ }

NB。记住你决定使用 do_something_with_this 尝试避免任何阻止。请参阅其他非阻塞 AnyEvent模块

NB. Remember whatever you decide todo with do_something_with_this try to avoid anything that blocks. See other non-blocking AnyEvent modules

/ I3az /

这篇关于什么是当前最舒适可靠的跨平台Perl模块来做并行下载？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！