内存使用情况和图像处理 [英] Memory usage and manipulating images

问题描述

TL; DR;转换为base64string的图像在大型对象堆中具有巨大的RAM占用空间.

TL;DR; Images converted to base64string have huge RAM footprint in large object heap.

我在Windows服务中有一些代码，可以使用用户上传的产品图片，将其标准化为网络级格式(它们将上传10MB位图)，并执行其他一些操作，例如将它们调整为正方形并添加空白填充.

I have some code in a windows service that consumes our product images uploaded by users, standardizes them into a web-grade format (they will upload 10MB bitmaps), and does some other things like resize them into a square and add whitespace padding.

然后将它们转换为base64字符串，以通过休息将其上传到我们的托管环境中.环境要求以这种方式完成操作，因此我无法使用URLS.当我这样做时，它们会随时间存储在大对象堆和程序的RAM使用率激增中.

It then converts them to a base64 string to upload them into our hosting environment via rest. The environment requires it be done this way, i cannot use URLS. When I do this, they get stored on the large object heap and the program's RAM usage skyrockets over time.

如何解决这个问题?

这是代码

private void HandleDocuments(IBaseProduct netforumProduct, MagentoClient client, bool isChild)
{
    if (netforumProduct.Documents == null) { return; }

    for (int idx = 0; idx < netforumProduct.Documents.Count; idx++)
    {
        JToken document = netforumProduct.Documents[idx]["Document"];
        if (document == null) { continue; }

        string fileName = document["URL"].ToString();

        // Skip photos on child products (the only identifier is part of the url string)
        if (fileName.ToLower().Contains("photo") && isChild) { continue; }

        using (HttpClient instance = new HttpClient {BaseAddress = client.NetforumFilesBaseAddress})
        {
            string trimStart = fileName.TrimStart('.');

            string base64String;

            using (Stream originalImageStream = instance.GetStreamAsync("iweb" + trimStart).Result)
            {
                using (MemoryStream newMemoryStream = new MemoryStream())
                {
                    using (Image img = Image.FromStream(originalImageStream))
                    {
                        using (Image retImg = Utility.Framework.ImageToFixedSize(img, 1200, 1200))
                        {
                            retImg.Save(newMemoryStream, ImageFormat.Jpeg);
                        }
                    }

                    newMemoryStream.Position = 0;

                    byte[] bytes = newMemoryStream.ToArray();
                    base64String = Convert.ToBase64String(bytes);
                }
            }

            // MediaGalleryEntry is a simple class with a few string properties
            MediaGalleryEntry mge = new MediaGalleryEntry
            {
                label = "Product_" + netforumProduct.Code + "_image_" + idx,
                content = new MediaGalleryContent
                {
                    base64_encoded_data = base64String,
                    name = "Gallery_Image_" + idx
                },
                file = trimStart
            };

            this.media_gallery_entries.Add(mge);
        }
    }
}

它不是有史以来最好的代码，可能没有经过高度优化，但是它是我能做到的最好的代码.

Its not the best code ever, probably not highly optimized, but its the best I can do.

推荐答案

TL; DR;转换为base64string的图像在大型对象堆中具有巨大的RAM占用空间

TL;DR; Images converted to base64string have huge RAM footprint in large object heap

是的，显然是这样.所有图像都是巨大的.压缩方法仅适用于存储和传输.但是，当图像加载到内存中以进行显示或进一步处理时，所有压缩步骤都必须撤消.这是与他们一起工作的人们的常见陷阱.

Yes, that is obviously true. All images are huge. Compression methods only apply to storage and transfer. But when the Image is loaded into memory - for display or further processing - all compression steps have to be undone. This is a common pitfall of people working with them.

然后将它们转换为Base64字符串，以通过休息将其上传到我们的托管环境中.环境要求以这种方式完成操作，因此我无法使用URLS.当我这样做时，它们会随着时间的流逝而存储在大对象堆和程序的RAM使用率激增中." Base64无效，但不会对此增加很多. + 25％IIRC.

It then converts them to a Base64 string to upload them into our hosting environment via rest. The environment requires it be done this way, i cannot use URLS. When I do this, they get stored on the large object heap and the program's RAM usage skyrockets over time." Base64 is ineffective, but will not add a lot to this. +25% IIRC.

主要问题是您是否真的在这里看到问题，还是只是误读了内存占用? @CodeCaster发现您保留了一个引用(这是一个实际的问题，并且是.NET中完全导致内存泄漏的几种方法之一)，但是即使您松开了这些字符串，该字符串仍将在内存中保留一段时间.

The big questions if you are really seeing an issue here, or are only misreading the memory footprint? @CodeCaster figured out that you kept a reference (wich is a real problem and one of the few ways you can get a memory leak in .NET at all), but even if you loose those this string will still stay in memory for some time.

.NET使用GarbageCollection内存管理方法.这种方法有一个问题:在收集GC的同时，必须暂停访问同一托管区域的所有其他线程.结果，由于缺少更好的用语，GC运行时非常懒惰.如果它仅在应用程序关闭时运行一次，那是理想的情况.可以让它更早运行的唯一原因是:

.NET uses the GarbageCollection Memory Management approach. That approach has one issue: While the GC collects, all other Threads accessing the same managed area have to be paused. As a result the GC is - for lack of a better term - very lazy with running. If it only runs once on application closure, that is the ideal situation. The only things that can get it to run earlier are:

• 调用GC.Collect();，通常不应在生产代码中使用它，仅在发生参考内存泄漏的情况下才用于调试
• OOM期望的危险
• 某些其他GC模式，例如服务器端之类的特殊东西

• calls to GC.Collect(); which should generally not be used in productive code, only for debugging if you got a reference memory leak
• the danger of a OOM Expection
• some of the alternative GC modes, particular stuff like the server one

我所能告诉您的是，它最终会运行 .但我认为您不必一定要知道确切的时间.

All I can tell you that it will run eventually. But I do not think you need to know the exact time necessarily.

这篇关于内存使用情况和图像处理的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！