从C ++更新Texture2D像素 [英] Update Texture2D pixels from C++

问题描述

在项目中，我正在通过DllImports使用Unity3D的C#脚本和C ++.我的目标是游戏场景中有2个立方体(Cube & Cube2)，其中一个立方体纹理通过我的便携式摄像机显示实况视频，而Unity的webCamTexture.Play()则另一个立方体纹理通过外部C ++函数ProcessImage()显示处理后的视频.

In the project, I'm using Unity3D's C# scripts and C++ via DllImports. My goal is that the game scene has 2 cubes (Cube & Cube2) which one texture of cube show the live video through my laptop camera and Unity's webCamTexture.Play() and the other one texture of cube show the processed video by external C++ function ProcessImage().

代码上下文:

在c ++中，我对此进行了定义

In c++, I define that

struct Color32
{
    unsigned char r;
    unsigned char g;
    unsigned char b;
    unsigned char a;
};

函数是

extern "C"
{
    Color32* ProcessImage(Color32* raw, int width, int height);
}
...

Color32* ProcessImage(Color32* raw, int width, int height)
{
    for(int i=0; i<width*height ;i++)
    {
       raw[i].r = raw[i].r-2;
       raw[i].g = raw[i].g-2;
       raw[i].b = raw[i].b-2;
       raw[i].a = raw[i].a-2;
    }
    return raw;
}

C#:

声明并导入

public GameObject cube; 
public GameObject cube2;
private Texture2D tx2D;
private WebCamTexture webCamTexture;

[DllImport("test22")]   /*the name of Plugin is test22*/
private static extern Color32[] ProcessImage(Color32[] rawImg, 
                                                 int width, int height);

获取相机情况并设置cube1，cube2纹理

Get camera situation and set cube1, cube2 Texture

void Start()
{
    WebCamDevice[] wcd = WebCamTexture.devices;

    if(wcd.Length==0)
    {
        print("Cannot find a camera");
        Application.Quit();
    }
    else
    {
        webCamTexture = new WebCamTexture(wcd[0].name);
        cube.GetComponent<Renderer>().material.mainTexture = webCamTexture;

        tx2D = new Texture2D(webCamTexture.width, webCamTexture.height);
        cube2.GetComponent<Renderer>().material.mainTexture = tx2D;

        webCamTexture.Play();
    }
}

通过DllImports将数据发送到外部C ++函数，并使用Color32[] a接收处理后的数据.最后，我使用Unity的SetPixels32设置tx2D(Cube2)纹理:

Send data to external C++ function by DllImports and receive processed data using Color32[] a. Finally, I'm using Unity's SetPixels32 to setup tx2D(Cube2) texture:

void Update()
{
    Color32[] rawImg = webCamTexture.GetPixels32();
    System.Array.Reverse(rawImg);

    Debug.Log("Test1");
    Color32[] a = ProcessImage(rawImg, webCamTexture.width, webCamTexture.height);
    Debug.Log("Test2");

    tx2D.SetPixels32(a);
    tx2D.Apply();
}

结果:

结果只是多维数据集1的纹理显示了实时视频，而无法使用多维数据集2的纹理显示处理后的数据.

The result is just the texture of cube 1 show the live video and fail to show processed data using the texture of cube 2.

错误:

SetPixels32调用，数组中的像素数无效 UnityEngine.Texture2D:SetPixels32(Color32 [])网络摄像头:Update()(在 资产/脚本/Webcam.cs:45)

SetPixels32 called with invalid number of pixels in the array UnityEngine.Texture2D:SetPixels32(Color32[]) Webcam:Update() (at Assets/Scripts/Webcam.cs:45)

我不明白为什么当我向SetPixels32输入数组a时数组中无效的像素数

I don't understand that why the invalid number of pixels in the array when I input array a to SetPixels32

有什么想法吗?

更新(2018年10月10日)

UPDATE(10 Oct. 2018)

感谢@Programmer，现在它可以通过引脚存储工作.

Thanks to @Programmer, now it can work by pin memory.

顺便说一句，我发现了一些问题，这是关于Unity Engine的.当Unity Camera在0到1秒之间运行时，webCamTexture.width或webCamTexture.height总是返回16x16尺寸，甚至要求更大的图像，例如1280x720，然后在1秒钟后返回正确的尺寸. (可能是几帧)因此，我引用了

Btw, I find some little problem which is about Unity Engine. When the Unity Camera run between 0 to 1 second, webCamTexture.width or webCamTexture.height always return 16x16 size even requested bigger image such as 1280x720 and then it will return correct size after 1 second. (Possibly several frames) So, I reference this post and delay 2 seconds to run Process() in Update() function and reset the Texture2D size in Process() function. It will work fine:

delaytime = 0;
void Update()
{
    delaytime = delaytime + Time.deltaTime;
    Debug.Log(webCamTexture.width);
    Debug.Log(webCamTexture.height);

    if (delaytime >= 2f)
        Process();
}
unsafe void Process()
{
    ...

    if ((Test.width != webCamTexture.width) || Test.height != webCamTexture.height)
    {
       Test = new Texture2D(webCamTexture.width, webCamTexture.height, TextureFormat.ARGB32, false, false);
       cube2.GetComponent<Renderer>().material.mainTexture = Test;
       Debug.Log("Fixed Texture dimension");
    }
    ...
}

推荐答案

C#不会返回了解您从C ++返回的Color32对象.如果您将返回类型设置为unsigned char*，然后在C#端使用Marshal.Copy将返回的数据复制到字节数组，然后使用此处是从C#返回字节数组的示例.

C# doesn't return understand the Color32 object you returned from C++. This would have worked if you made the return type to be unsigned char* then use Marshal.Copy on the C# side to copy the returned data into byte array then use Texture2D.LoadRawTextureData to load the array into your Texture2D. Here is an example that returns byte array from C#.

我不建议您返回数据，因为这很昂贵.填充要传递给函数的数组，然后只需将该数组重新分配给您的Texture2D.

I wouldn't suggest you return data as that's costly. Fill the array you are passing to the function then simply re-assign that array to your Texture2D.

C ++:

extern "C"
{
    void ProcessImage(unsigned char* raw, int width, int height);
}
...

void ProcessImage(unsigned char* raw, int width, int height)
{
    for(int y=0; y < height; y++)
    {
        for(int x=0; x < width; x++)
        {
            unsigned char* pixel = raw + (y * width * 4 + x * 4);
            pixel[0] = pixel[0]-(unsigned char)2; //R
            pixel[1] = pixel[1]-(unsigned char)2; //G
            pixel[2] = pixel[2]-(unsigned char)2; //B
            pixel[3] = pixel[3]-(unsigned char)2; //Alpha
        }
    }
}

C#:

[DllImport("test22")]
private static extern void ProcessImage(IntPtr texData, int width, int height);

unsafe void ProcessImage(Texture2D texData)
{
    Color32[] texDataColor = texData.GetPixels32();
    System.Array.Reverse(texDataColor);

    //Pin Memory
    fixed (Color32* p = texDataColor)
    {
        ProcessImage((IntPtr)p, texData.width, texData.height);
    }
    //Update the Texture2D with array updated in C++
    texData.SetPixels32(texDataColor);
    texData.Apply();
}

要使用:

public Texture2D tex;

void Update()
{
    ProcessImage(tex);
}

如果不想使用unsafe和fixed关键字，还可以使用GCHandle.Alloc固定数组，然后使用GCHandle.AddrOfPinnedObject将其发送到C ++端.有关如何操作的信息，请参见此帖子.

If you don't want to use the unsafe and fixed keywords, you can also use GCHandle.Alloc to pin the array before sending it to the C++ side with GCHandle.AddrOfPinnedObject. See this post for how to do that.

如果遇到以下异常:

SetPixels32调用，数组中的像素数无效 UnityEngine.Texture2D:SetPixels32(Color32 [])网络摄像头:Update()(在 资产/脚本/Webcam.cs:45)

SetPixels32 called with invalid number of pixels in the array UnityEngine.Texture2D:SetPixels32(Color32[]) Webcam:Update() (at Assets/Scripts/Webcam.cs:45)

这意味着您正在调用SetPixels32的Texture2D具有与WebCamTexture纹理不同的大小或纹理尺寸.要解决此问题，请在调用SetPixels32之前，检查纹理尺寸是否已更改，然后调整目标纹理的大小以与WebCamTexture匹配.

This means that the Texture2D you're calling SetPixels32 on has different size or texture dimension with the WebCamTexture texture. To fix this, before calling SetPixels32, check if the texture dimension changed then resize your target texture to match with the WebCamTexture.

替换

tx2D.SetPixels32(rawImg);
tx2D.Apply();

使用

//Fix texture dimension if it doesn't match with the web cam texture size
if ((tx2D.width != webCamTexture.width) || tx2D.height != webCamTexture.height)
{
    tx2D = new Texture2D(webCamTexture.width, webCamTexture.height, TextureFormat.RGBA32, false, false);
    Debug.Log("Fixed Texture dimension");
}

tx2D.SetPixels32(rawImg);
tx2D.Apply();

这篇关于从C ++更新Texture2D像素的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！