将图像从统一传递到opencv [英] Passing Images to opencv from unity

问题描述

我创建了一个dll，以将两个图像从unity传递到opencv并返回一个struct.但是我遇到了运行时C ++错误. 这是我在C ++中使用的代码:

I've created a dll to pass two images from unity to opencv and return a struct. But I'm getting a run time c++ error. This is the code I used in C++:

    struct rigidTransform
    {
        rigidTransform(double eular_angle_x, double eular_angle_y, double eular_angle_z, double eular_dis_x, double eular_dis_y, double eular_dis_z) : Eular_angle_x(eular_angle_x), Eular_angle_y(eular_angle_y), Eular_angle_z(eular_angle_z), Eular_dis_x(eular_dis_x), Eular_dis_y(eular_dis_y), Eular_dis_z(eular_dis_z) {}
        double Eular_angle_x;
        double Eular_angle_y;
        double Eular_angle_z;

        double Eular_dis_x;
        double Eular_dis_y;
        double Eular_dis_z;
    };
        struct Color32
    {
        uchar r;
        uchar g;
        uchar b;
        uchar a;
    };

    extern "C" rigidTransform __declspec(dllexport) __stdcall findPose(Color32* img1, Color32* img2,  int width, int height)
    {
    Mat Img1(height, width, CV_8UC4, img1), Img2(height, width, CV_8UC4, img2);
    //my pose estimation code here

    return Tr;
    }

我统一使用的C#代码是:

And my C# code used in unity is:

    using System.Collections;
    using System.Collections.Generic;
    using System.Runtime.InteropServices;
    using UnityEngine;

    public struct regidTransform
    {
        public double eular_angle_x;
        public double eular_angle_y;
        public double eular_angle_z;
        public double eular_dis_x;
        public double eular_dis_y;
        public double eular_dis_z;
    }



public class UI : MonoBehaviour
{
    [DllImport("test2", EntryPoint = "findPose", CallingConvention = CallingConvention.Cdecl)]
    public unsafe static extern regidTransform findPose(Color32[] img1, Color32[] img2, int width, int height);

    public regidTransform results_;


    public WebCamTexture webcamTexture;
    public Color32[] img1;
    public Color32[] img2;

    void Start()
    {

        results_ = new regidTransform();




        webcamTexture = new WebCamTexture();
        webPlane.GetComponent<MeshRenderer>().material.mainTexture = webcamTexture;
        webcamTexture.Play();

        if(webcamTexture.isPlaying)
        {
            img1 = webcamTexture.GetPixels32();
            System.Array.Reverse(img1);
            img2 = webcamTexture.GetPixels32();
            System.Array.Reverse(img2);


            unsafe
            {

                    results_ = findPose(img1, img2, webcamTexture.width, webcamTexture.height);


            }
        }




    }


    void Update()
    {


    }
}

当我运行这段代码时，它抛出一个错误:

When I run this code, it throws an error saying:

运行时错误！

Runtime Error!

程序:

此应用程序已请求运行时以终止运行时 不寻常的方式.请与应用程序支持团队联系.

This Application has requested the Runtime to terminate it in an unusual way. Please contact the application support team.

请协助我解决这段代码中的错误.

Kindly assist me on what wrong I've made in this code.

先谢谢您！

已更新:

在@Programmer的帮助下，我发现调用calcOpticalFlowPyrLK()时代码崩溃了.并且，当我使用imshow()功能检查是否可以正确获取图像时，我发现它会产生完整的黑色图像.

With the help of @Programmer I found that the code is getting crashed when calling calcOpticalFlowPyrLK(). And when checked if I'm getting the image correctly using imshow() function, I found it results a complete black image.

推荐答案

最后我弄清楚了.

是的，代码运行良好.并且我的C ++代码或C#代码没有错误.对我来说导致完整的黑色图像的原因是，我将dll功能称为统一的strat()函数，仅执行一次.因此，在我的案例中，我发现，获取到dll的典型第1张图像大部分将是黑色的，这是在dll中使用imshow()函数进行测试时得到证明的.引发错误的原因是因为我将黑色图像传递给了calcOpticalFlowPyrLK()函数，该函数没有任何要跟踪的特征点.

Yes that code worked well. and there is no error in my C++ code or C# code. The reason that resulted a complete black image for me is, I call my dll fuction in side strat() function of unity, which only execute once. So I found in my case, that the typical 1st image acquired to dll will mostly a black one, this was proved when I tested with imshow() function inside the dll. The reason it throws error is because I'm passing that black image to calcOpticalFlowPyrLK() function, which doesn't have any feature points to track.

因此，我在dll中创建了另一个函数，以检查是否已获取真实图像，然后再通过检查特征点将其传递给我的代码.

So I created another function inside dll to check if the true image is acquired before passing it to my code by checking for feature points.

类似的东西:

featureDetection(frame_1, points1);

if (points1.size() > MIN_NUM_FEATURES)
{
    //Pass the frame to my code.
}

现在一切都会正常进行.我衷心感谢@Programmer帮助我弄清楚了这一点.

Now things will perfectly work. My sincere thanks to the @Programmer who helped me to figure this out.

这篇关于将图像从统一传递到opencv的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！