PHP中的大量图像下载脚本 [英] Mass image download script in PHP

问题描述

我需要一个.php脚本，它将从另一个站点下载很多图像。图像是大拇指 - 每个都有大约20KB的大小。
我已经在自己的脚本上工作，但是遗憾的是它只是滞后于我的服务器，几乎杀死了它迫使我重新启动它。

I need a .php script that will download a lot of images from another site. The images are thumbs - each has about 20KB size. I have worked on my own script, but sadly it just lags my server and nearly kills it forcing me to restart it.

有大约100张照片或更多的每个执行，.jpg文件，〜20KB /文件。

There are about 100 pictures or more per execution, .jpg files, ~20KB / file.

我的脚本：

$count = 0;
foreach ($files as $file) {
$count++;
$url = $file;
$dl_place = '/home/lulz/'.$count.'.jpg';

$ch = curl_init($dl);
$fp = fopen($path, 'wb');
curl_setopt($ch, CURLOPT_FILE, $fp);
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_exec($ch);
curl_close($ch);
fclose($fp);
}

正如你所看到的，我使用卷曲，但是我愿意使用任何东西它的效果比现在更好。

As you see I am using curl, but I am willing to use anything if it just works better than it is now.

推荐答案

有可能是什么是放缓是设置所有时间的这些请求。您应该考虑并行cURL 一次下载多个。源代码： https://github.com/petewarden/ParallelCurl/blob/master/ parallelcurl.php

Chances are, what is slowing down is the time it takes to set up all of these requests. You should consider Parallel cURL to download multiple at a time. Source code: https://github.com/petewarden/ParallelCurl/blob/master/parallelcurl.php

$pc->startRequest('http://www.whatever.com/someimage.jpg', 'your_callback_function');

我还发现使用库，您可以使用匿名函数而不是函数的名称你的回调例如，我使用它来调用另一个带有ID号的函数。

I have also found that with library, you can use anonymous functions instead of the name of a function in your callback. I use this to call another function with an ID number, for example.

$requestid=37;
$pc->startRequest(
    $url, 
    function($content, $url, $ch, $search) use $requestid {
        yourRealCallback($content, $url, $ch, $search, $requestid);
    }
);

这个函数利用了一个封闭的匿名函数，如果你正在搜索一个数据库的URL，你可以得到生成的ID（您在for循环中指定或某些...硬编码为'37'，用于演示）。

This utilizes an anonymous function with closure so that if you are searching a DB of URLs, you can get the resulting ID (that you specify in a for loop or something... hard-coded to '37' here for demonstration).

这篇关于PHP中的大量图像下载脚本的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！