加载大图像时达到硬盘限制，加载时可能降采样吗? [英] Hitting hard memory limit when loading large image, possible to downsample while loading?

问题描述

我需要从可能巨大的图像(10,000 x 10,000)中生成小的缩略图.我正在使用在Google App Engine中运行的ImageMagick和PHP.我的GAE的硬盘内存限制为512 MB.

I need to generate small thumbnails from potentially gigantic images (10,000 x 10,000). I'm using ImageMagick and PHP running in Google App Engine. My GAE has a hard memory limit of 512 MB.

我一直在尝试阅读有关imagemagick php扩展的文档，但是文档非常薄.我发现了这篇SO帖子，其中包含维护者的答案，并尝试了适应他们的代码.我试图设置资源区域，以便将大于2000x2000像素的图像换出到磁盘上，而不是保存在内存中，但是我误解了代码或做错了事.我的GAE仍然崩溃，说我已经填满了内存.

I've been trying to read the documentation for the imagemagick php extension, but the docs are pretty thin. I found this SO post that has an answer from the maintainer and tried to adapt their code. I tried to set the resource area so that images which were larger than 2000x2000 pixels would get swapped out to disk instead of held in memory, but either I misunderstand the code or I'm doing something wrong. My GAE still crashes saying that I've filled the memory.

try {
    Imagick::setResourceLimit(Imagick::RESOURCETYPE_AREA, 2000 * 2000);
    Imagick::setResourceLimit(Imagick::RESOURCETYPE_DISK, 1024*1024*1024*2);

    $im = new \Imagick($path);
    
    // ...

} catch (\Exception $e) {
    error_log("Exception: " . $e->getMessage() . "\n");
}

超出了512 MB的512 MB硬盘限制...

在Android设备上，当从磁盘加载位图时，有一个选项可以在将其加载到内存时对其进行下采样.<​​/p一个>.这样一来，我便可以将较小版本的图像加载到内存中，避免了巨大的文件破坏所有内容的问题.

On Android devices, when loading a Bitmap from disk, there is an option to downsample it as it is being loaded into memory. That allows me to load smaller versions of the image into memory and avoid the problem of gigantic files breaking everything.

ImageMagick中是否有类似的东西?我看到了 ImageMagick :: setSamplingFactors()，但文档没有完全解释这些参数，所以我不确定如何使用它.

Is there something similar in ImageMagick? I saw ImageMagick::setSamplingFactors() but the documentation doesn't explain the parameters at all so I'm not sure how to use it.

如何在不达到Google App Engine的硬盘限制的情况下从巨型图像生成微小的缩略图?

How can I generate a tiny thumbnail from a giant image without hitting my hard memory limit on Google App Engine?

推荐答案

如果要收缩JPEG图像，则imagemagick支持加载时收缩.例如，这是一个10k x 10k像素的JPEG图像，尺寸缩小到200x200.

If you are shrinking JPEG images, then imagemagick supports shrink-on-load. For example, here's a 10k x 10k pixel JPEG image being sized down to 200x200.

$ /usr/bin/time -f %M:%e \
    convert wtc.jpg -resize 200x200 x.jpg
713340:2.98

这是720MB的峰值内存使用量和将近3s的CPU时间.现在尝试这个:

That's 720MB of peak memory use and almost 3s of CPU time. Now try this:

$ /usr/bin/time -f %M:%e \
    convert -define jpeg:size=400x400 wtc.jpg -resize 200x200 x.jpg
35952:0.32

最多35MB的内存和300ms的CPU.

Down to 35MB of memory and 300ms of CPU.

-define jpeg:size = 400x400 提示JPEG加载器您需要至少400x400像素的图像，因此(在这种情况下)加载时，它将以1/第8个尺寸.您需要使加载提示大小至少比最终输出大小大2倍，以免发生混淆.

The -define jpeg:size=400x400 hints to the JPEG loader that you want an image of at least 400x400 pixels, so (in this case) during load, it'll fetch at 1/8th size. You need the load hint size to be at least 2x larger than your final output size to avoid aliasing.

您可以使用 setOption 从imagick进行设置.

You can set this from imagick with setOption.

不幸的是，许多装载机不支持装载收缩.PNG尤其糟糕:

Unfortunately, many loaders do not support shrink-on-load. PNG is especially bad:

$ /usr/bin/time -f %M:%e \
    convert wtc.png -resize 200x200 x.jpg
828376:5.62

830MB和5.6s.

830MB and 5.6s.

您可以考虑其他调整大小的程序. vipsthumbnail 很快和几乎所有文件格式的低内存，例如:

You could consider other resize programs. vipsthumbnail is fast and low-memory for almost all file formats, for example:

$ /usr/bin/time -f %M:%e \
    vipsthumbnail wtc.png --size 200x200 -o x.jpg
58780:2.29

相同的PNG文件

60MB和2.3s.质量与imagemagick相同.

60MB and 2.3s for the same PNG file. Quality is the same as imagemagick.

它也具有 PHP绑定-您可以编写例如:

It has a PHP binding too -- you can write eg.:

$image = Vips\Image::thumbnail('somefile.jpg', 200);
$image->writeToFile('tiny.jpg');

这篇关于加载大图像时达到硬盘限制，加载时可能降采样吗?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！