php从url获得所有图像,宽度和高度> = 200更快 [英] php get all the images from url which width and height >=200 more quicker
问题描述
我模拟了某些功能,例如 http://pinterest.com add a pin
I am simular some function like http://pinterest.com add a pin
如何从宽度和高度> = 200的url中更快地获取所有图像? pinterest.com
将完成整个过程近10秒,但我需要48.64秒.
How to get all the images from url which width and height >=200 more quicker? pinterest.com
will finish the whole process nearly 10 seconds, but I need 48.64 seconds.
require dirname(__FILE__) . '/simple_html_dom.php';
$url = 'http://www.huffingtonpost.com/';
$html = file_get_html($url);
if($html->find('img')){
foreach($html->find('img') as $element) {
$size = @getimagesize($element->src);
if($size[0]>=200&&$size[1]>=200){
echo $element;
}
}
}// cost 48.64 seconds
推荐答案
我认为您使用的是使用curl_multi_init
在parallel
中运行curl
请求,请参见
I think what you use do is run curl
requests in parallel
using curl_multi_init
please see http://php.net/manual/en/function.curl-multi-init.php for more information. This way it will load much faster and escape all bandwidth issue that can also affect speed.
将映像保存到本地临时目录中,而不直接在本地运行getimagesize()
,这比通过http://
Save the image into a local temp directory not run getimagesize()
on the local directly which is much faster than running it over http://
我希望这对您有帮助
编辑1
注意***
A.并非所有图像都以http
A. Not all Images start with http
B.并非所有图片都是有效的
B. Not all images are valid
C.在需要存储图像的temp
文件夹中创建
C. Create temp
folder where the images needs to be stored
概念验证
require 'simple_html_dom.php';
$url = 'http://www.huffingtonpost.com';
$html = file_get_html ( $url );
$nodes = array ();
$start = microtime ();
$res = array ();
if ($html->find ( 'img' )) {
foreach ( $html->find ( 'img' ) as $element ) {
if (startsWith ( $element->src, "/" )) {
$element->src = $url . $element->src;
}
if (! startsWith ( $element->src, "http" )) {
$element->src = $url . "/" . $element->src;
}
$nodes [] = $element->src;
}
}
echo "<pre>";
print_r ( imageDownload ( $nodes, 200, 200 ) );
echo "<h1>", microtime () - $start, "</h1>";
function imageDownload($nodes, $maxHeight = 0, $maxWidth = 0) {
$mh = curl_multi_init ();
$curl_array = array ();
foreach ( $nodes as $i => $url ) {
$curl_array [$i] = curl_init ( $url );
curl_setopt ( $curl_array [$i], CURLOPT_RETURNTRANSFER, true );
curl_setopt ( $curl_array [$i], CURLOPT_USERAGENT, 'Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1.2) Gecko/20090729 Firefox/3.5.2 (.NET CLR 3.5.30729)' );
curl_setopt ( $curl_array [$i], CURLOPT_CONNECTTIMEOUT, 5 );
curl_setopt ( $curl_array [$i], CURLOPT_TIMEOUT, 15 );
curl_multi_add_handle ( $mh, $curl_array [$i] );
}
$running = NULL;
do {
usleep ( 10000 );
curl_multi_exec ( $mh, $running );
} while ( $running > 0 );
$res = array ();
foreach ( $nodes as $i => $url ) {
$curlErrorCode = curl_errno ( $curl_array [$i] );
if ($curlErrorCode === 0) {
$info = curl_getinfo ( $curl_array [$i] );
$ext = getExtention ( $info ['content_type'] );
if ($info ['content_type'] !== null) {
$temp = "temp/img" . md5 ( mt_rand () ) . $ext;
touch ( $temp );
$imageContent = curl_multi_getcontent ( $curl_array [$i] );
file_put_contents ( $temp, $imageContent );
if ($maxHeight == 0 || $maxWidth == 0) {
$res [] = $temp;
} else {
$size = getimagesize ( $temp );
if ($size [1] >= $maxHeight && $size [0] >= $maxWidth) {
$res [] = $temp;
} else {
unlink ( $temp );
}
}
}
}
curl_multi_remove_handle ( $mh, $curl_array [$i] );
curl_close ( $curl_array [$i] );
}
curl_multi_close ( $mh );
return $res;
}
function getExtention($type) {
$type = strtolower ( $type );
switch ($type) {
case "image/gif" :
return ".gif";
break;
case "image/png" :
return ".png";
break;
case "image/jpeg" :
return ".jpg";
break;
default :
return ".img";
break;
}
}
function startsWith($str, $prefix) {
$temp = substr ( $str, 0, strlen ( $prefix ) );
$temp = strtolower ( $temp );
$prefix = strtolower ( $prefix );
return ($temp == $prefix);
}
输出
Array
(
[0] => temp/img8cdd64d686ee6b925e8706fa35968da4.gif
[1] => temp/img5811155f8862cd0c3e2746881df9cd9f.gif
[2] => temp/imga597bf04873859a69373804dc2e2c27e.jpg
[3] => temp/img0914451e7e5a6f4c883ad7845569029e.jpg
[4] => temp/imgb1c8c4fa88d0847c99c6f4aa17a0a457.jpg
[5] => temp/img36e5da68a30df7934a26911f65230819.jpg
[6] => temp/img068c1aa705296b38f2ec689e5b3172b9.png
[7] => temp/imgfbeca2410b9a9fb5c08ef88dacd46895.png
)
0.076347
谢谢 :)
这篇关于php从url获得所有图像,宽度和高度> = 200更快的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!