在C#中调整图像大小并将其发送到OpenCV会导致图像失真 [英] Resizing the image in C# and sending it to OpenCV results in distorted image

问题描述

这是与

This is a follow-up question related to this one. Basically, I have a DLL which uses OpenCV to do image manipulation. There are two methods, one accepting an image-Path, and the other one accepting a cv::Mat. The one working with image-path works fine. The one that accepts an image is problematic.

以下是接受文件名(DLL)的方法:

Here is the method which accepts the filename (DLL):

CDLL2_API void Classify(const char * img_path, char* out_result, int* length_of_out_result, int N)
{
    auto classifier = reinterpret_cast<Classifier*>(GetHandle());

    cv::Mat img = cv::imread(img_path);
    cv::imshow("img recieved from c#", img);
    std::vector<PredictionResults> result = classifier->Classify(std::string(img_path), N);
    std::string str_info = "";
    //...
    *length_of_out_result = ss.str().length();
}

这是接受图像(DLL)的方法:

Here is the method which accepts the image (DLL):

CDLL2_API void Classify_Image(unsigned char* img_pointer, unsigned int height, unsigned int width, 
                              int step, char* out_result, int* length_of_out_result, int top_n_results)
    {
        auto classifier = reinterpret_cast<Classifier*>(GetHandle());
        cv::Mat img = cv::Mat(height, width, CV_8UC3, (void*)img_pointer, step);
        std::vector<Prediction> result = classifier->Classify(img, top_n_results);

        //...
        *length_of_out_result = ss.str().length();
    }

这是C#应用程序中的代码:DllImport:

Here is the code in C# application: DllImport:

[DllImport(@"CDll2.dll", CallingConvention = CallingConvention.Cdecl, CharSet = CharSet.Ansi)]
        static extern void Classify_Image(IntPtr img, uint height, uint width, int step, byte[] out_result, out int out_result_length, int top_n_results = 2);

将图像发送到DLL的方法:

The method which sends the image to the DLL:

private string Classify_UsingImage(Bitmap img, int top_n_results)
{
    byte[] res = new byte[200];
    int len;
    BitmapData bmpData;
    bmpData = img.LockBits(new Rectangle(0, 0, img.Width, img.Height), ImageLockMode.ReadOnly, img.PixelFormat);
    Classify_Image(bmpData.Scan0, (uint)bmpData.Height, (uint)bmpData.Width, bmpData.Stride, res, out len, top_n_results);
    //Remember to unlock!!!
    img.UnlockBits(bmpData); 
    string s = ASCIIEncoding.ASCII.GetString(res);
    return s;
}

现在，当我将图像发送到DLL时，此方法效果很好.如果我使用imshow()来显示接收到的图像，则图像显示得很好.

Now, this works well when I send an image to the DLL. if I use imshow() to show the received image, the image is shown just fine.

实际问题:

但是，当我调整大小并使用上述完全相同的方法发送图像时，图像会失真.

However, when I resize the very same image and send it using the very same method above, the image is distorted.

我需要补充一点，如果我使用下面的给定C#方法调整图像的大小，然后保存它，然后将文件名传递给DLL，然后使用Classify(std::string(img_path), N);打开它，则效果很好.

I need to add that, If I resize an image using the given C# method below, then save it, and then pass the filename to the DLL to be opened using Classify(std::string(img_path), N); it works perfectly.

以下是屏幕快照，显示了这种情况的示例:
从C`发送而未调整大小的图像:

Here is the screenshot showing an example of this happening:
Image sent from C` without being resized:

首先调整同一图像的大小，然后将其发送到DLL:

When The same image is first resized and then sent to the DLL:

在此首先将图像调整大小(用C#格式)，保存到磁盘，然后将其filepath发送到DLL:

Here the image is first resized (in C#), saved to the disk and then its filepath sent to the DLL:

这是负责调整大小(C#)的代码段:

This is the snippet responsible for resizing (C#):

/// <summary>
/// Resize the image to the specified width and height.
/// </summary>
/// <param name="image">The image to resize.</param>
/// <param name="width">The width to resize to.</param>
/// <param name="height">The height to resize to.</param>
/// <returns>The resized image.</returns>
public static Bitmap ResizeImage(Image image, int width, int height)
{
    var destRect = new Rectangle(0, 0, width, height);
    var destImage = new Bitmap(width, height);

    destImage.SetResolution(image.HorizontalResolution, image.VerticalResolution);

    using (var graphics = Graphics.FromImage(destImage))
    {
        graphics.CompositingMode = CompositingMode.SourceCopy;
        graphics.CompositingQuality = CompositingQuality.HighQuality;
        graphics.InterpolationMode = InterpolationMode.HighQualityBicubic;
        graphics.SmoothingMode = SmoothingMode.HighQuality;
        graphics.PixelOffsetMode = PixelOffsetMode.HighQuality;

        using (var wrapMode = new ImageAttributes())
        {
            wrapMode.SetWrapMode(WrapMode.TileFlipXY);
            graphics.DrawImage(image, destRect, 0, 0, image.Width, image.Height, GraphicsUnit.Pixel, wrapMode);
        }
    }

    return destImage;
}

这是原始图像

这是Classify方法，该方法使用文件路径读取图像:

This is the Classify method which uses the filepath to read the images:

std::vector<PredictionResults> Classifier::Classify(const std::string & img_path, int N)
{
    cv::Mat img = cv::imread(img_path);
    cv::Mat resizedImage;
    std::vector<PredictionResults> results;
    std::vector<float> output;

   // cv::imshow((std::string("img classify by path") + type2str(img.type())), img);
    if (IsResizedEnabled())
    {
        ResizeImage(img, resizedImage);
        output = Predict(resizedImage);
    }
    else
    {
        output = Predict(img);
        img.release();
    }

    N = std::min<int>(labels_.size(), N);
    std::vector<int> maxN = Argmax(output, N);
    for (int i = 0; i < N; ++i)
    {
        int idx = maxN[i];
        PredictionResults r;
        r.label = labels_[idx];
        r.accuracy = output[idx];
        results.push_back(r);
    }

    return results;
}

这是上面方法中使用的ResizeImage:

And this is the ResizeImage used in the method above :

void Classifier::ResizeImage(const cv::Mat & source_image, cv::Mat& resizedImage)
{
    Size size(GetResizeHeight(), GetResizeHeight());
    cv::resize(source_image, resizedImage, size);//resize image

    CHECK(!resizedImage.empty()) << "Unable to decode image ";
}

问题2:
除了调整大小后出现失真之外，我面临着在C#中调整大小与使用OpenCV本身进行调整之间的差异.
我使用EmguCV创建了另一种方法(也在下面给出)，并传递了所需的信息，并且在我们使用C#调整图像大小并将其发送到DLL时，没有遇到任何此类失真.
但是，这种差异让我想了解是什么原因导致了这些问题.
这是使用EmguCV.Mat的方法，该方法无论大小如何都可以工作:

Problem 2:
Distortion after resizing aside, I am facing a discrepancy between resizing in C# and resizing using OpenCV itself.
I have created another method using EmguCV (also given below) and passed the needed information and did not face any such distortions which happen when we resize the image in C# and send it to the DLL.
However, this discrepancy made me want to understand what is causing these issues.
Here is the method which uses EmguCV.Mat is the code that works irrespective of resizing:

private string Classify_UsingMat(string imgpath, int top_n_results)
{
    byte[] res = new byte[200];
    int len;

    Emgu.CV.Mat img = new Emgu.CV.Mat(imgpath, ImreadModes.Color);

    if (chkResizeImageCShap.Checked)
    {
        CvInvoke.Resize(img, img, new Size(256, 256));
    }

    Classify_Image(img.DataPointer, (uint)img.Height, (uint)img.Width, img.Step, res, out len, top_n_results);

    string s = ASCIIEncoding.ASCII.GetString(res);
    return s;
}

我为什么在乎?
因为，当我使用OpenCV调整大小(使用EMguCV的CvInvoke.resize()和cv::resize())时，获得的精度与在C#中调整图像大小，将其保存到磁盘并将图像路径发送到openCV时获得的精度不同.
因此，我要么需要解决在C#中处理图像时发生的变形，要么需要理解为什么OpenCV中的大小调整与C#大小调整具有不同的结果.

Why do I care?
Because, I get a different accuracy when I use OpenCV resize (both when I use EMguCV's CvInvoke.resize() and cv::resize()) than what I get from resizing the image in C#, saving it to disk and send the image path to the openCV.
So I either need to fix the distortion happening when I deal with images in C#, or I need to understand why the resizing in OpenCV has different results than the C# resizing.

因此，总结到目前为止的问题和要点:

So to summarize issues and points made so far:

1. 所有情况都完整无缺，如果我们在C#应用程序中调整图像的大小， 并像以前一样正常传递信息，图像将是 扭曲(上面给出的示例)
2. 如果我们调整图像的大小，请将其保存到磁盘，并提供其文件名 到OpenCV来创建新的cv::Mat，它可以完美运行而没有任何问题.

3. 如果我使用EmugCV，而不是使用Bitmap，请使用 Emug.CV.Mat并使用来自以下对象的mat对象发送所需的参数 C#，不会发生扭曲.

1. All situations intact, If we resize the image inside C# application, and pass the info normally as we did before, the image will be distorted (example given above)
2. If we resize the image, save it to the disk, and give its filename to the OpenCV to create a new cv::Mat, it works perfectly without any issues.

3. If I use EmugCV and instead of working with Bitmap, use Emug.CV.Mat and send the needed parameters using mat object from C#, no distortion happens.

但是，我从C#调整大小后的图像中获得的准确度(请参阅#2)与使用OpenCV从调整大小后的图像中获得的精度有所不同. 如果我在使用之前手动调整图像大小，这没有任何区别 CvInvoke.Resize()从C#，并将结果图像发送到DLL， 或发送原始图像(使用EmguCV)并使用cv::resize()在C ++代码中调整其大小.这就是阻止我使用EmguCV或传递图像原始图像并使用OpenCV在DLL内调整其大小的原因.

However, the accuracy I get from a resized image from C# (see #2), is different than the one I get from the resized image using OpenCV. This doesn't make any difference if I resize the image before hand using CvInvoke.Resize() from C#, and send the resulting image to the DLL, or send the original image (using EmguCV) and resizing it in the C++ code using cv::resize(). This is what prevents me from using the EmguCV or passing the image original image and resizing it inside the DLL using OpenCV.

以下是具有不同结果的图像，显​​示了这些问题:

Here are the images with different results, showing the issues:

--------------------No Resizing------------------------------
1.Using Bitmap-No Resize, =>safe, acc=580295
2.Using Emgu.Mat-No Resize =>safe, acc=0.580262
3.Using FilePath-No Resize, =>safe, acc=0.580262
--------------------Resize in C#------------------------------
4.Using Bitmap-CSharp Resize, =>unsafe, acc=0.770425
5.Using Emgu.Mat-EmguResize, =>unsafe, acc=758335
6.**Using FilePath-CSharp Resize, =>unsafe, acc=0.977649**
--------------------Resize in DLL------------------------------
7.Using Bitmap-DLL Resize, =>unsafe, acc=0.757484
8.Using Emgu.DLL Resize, =>unsafe, acc=0.758335
9.Using FilePath-DLL Resize, =>unsafe, acc=0.758335

我需要获得在#6处获得的准确度.如您所见，EmguCV调整大小以及DLL中使用的OpenCV调整大小功能，其行为类似且无法按预期工作(即像#2)！，应用于图像的C#调整大小方法存在问题，而如果调整大小，保存并传递文件名，结果会很好.

I need to get the accuracy which I get at #6. as you can see the EmguCV resize and also the OpenCV resize function used in the DLL, act similar and don't work as expected (i.e. like #2)!, The C# resize method applied on the image is problematic, while if it is resized, saved and the filename passed, the result will be fine.

您可以在此处看到描绘不同情况的屏幕截图: http://imgur.com/a/xbgIQ

You can see the screen shots depicting different scenarios here: http://imgur.com/a/xbgIQ

推荐答案

我所做的是使用imdecode 按照EdChum的建议. 这就是DLL和C#中的函数的外观:

What I did was to use imdecode as EdChum suggested. This is how the functions in the DLL and C# look now:

#ifdef CDLL2_EXPORTS
#define CDLL2_API __declspec(dllexport)
#else
#define CDLL2_API __declspec(dllimport)
#endif

#include "classification.h" 
extern "C"
{
    CDLL2_API void Classify_Image(unsigned char* img_pointer, long data_len, char* out_result, int* length_of_out_result, int top_n_results = 2);
//...
}

实际方法:

CDLL2_API void Classify_Image(unsigned char* img_pointer, long data_len,
                              char* out_result, int* length_of_out_result, int top_n_results)
{
    auto classifier = reinterpret_cast<Classifier*>(GetHandle());
    vector<unsigned char> inputImageBytes(img_pointer, img_pointer + data_len);
    cv::Mat img = imdecode(inputImageBytes, CV_LOAD_IMAGE_COLOR);

    cv::imshow("img just recieved from c#", img);

    std::vector<Prediction> result = classifier->Classify(img, top_n_results);
    //...
    *length_of_out_result = ss.str().length();
}

这是C#DllImport:

Here is the C# DllImport:

[DllImport(@"CDll2.dll", CallingConvention = CallingConvention.Cdecl, CharSet = CharSet.Ansi)]
static extern void Classify_Image(byte[] img, long data_len, byte[] out_result, out int out_result_length, int top_n_results = 2);

这是将图像发送回DLL的实际方法:

and this is the actual method sending the image back to the DLL:

private string Classify_UsingImage(Bitmap image, int top_n_results)
{
    byte[] result = new byte[200];
    int len;
    Bitmap img;

    if (chkResizeImageCShap.Checked)
        img = ResizeImage(image, int.Parse(txtWidth.Text), (int.Parse(txtHeight.Text)));
    else
        img = image;

    //this is for situations, where the image is not read from disk, and is stored in the memort(e.g. image comes from a camera or snapshot)
    ImageFormat fmt = new ImageFormat(image.RawFormat.Guid);
    var imageCodecInfo = ImageCodecInfo.GetImageEncoders().FirstOrDefault(codec => codec.FormatID == image.RawFormat.Guid);
    if (imageCodecInfo == null)
    {
        fmt = ImageFormat.Jpeg;
    }

    using (MemoryStream ms = new MemoryStream())
    {
        img.Save(ms, fmt);
        byte[] image_byte_array = ms.ToArray();
        Classify_Image(image_byte_array, ms.Length, result, out len, top_n_results);
    }

    return ASCIIEncoding.ASCII.GetString(result);
}

通过从C#调整图像大小后执行此操作，我们根本不会遇到任何失真.

By doing this after resizing the image from C#, we don't face any distortions at all.

但是，我无法弄清楚为什么在OpenCV部分上调整大小无法正常工作！

I couldn't, however, figure out why the resize on OpenCV part wouldn't work as expected!

这篇关于在C#中调整图像大小并将其发送到OpenCV会导致图像失真的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！