Android插件中的SurfaceTexture在Unity中不起作用 [英] SurfaceTexture in Android plugin doesn't work in Unity

问题描述

我无法将纹理绑定到要在Unity中显示的SurfaceTexture.

I can't get the texture tied to a SurfaceTexture to display in Unity.

更新4:基于更新1中的管道(表面->通过表面纹理的外部纹理-> fbo->纹理2d)，我知道SurfaceTexture不能正确转换其表面到纹理.我可以通过pixelcopy从其表面正确绘制图片，并且可以确认我的FBO到texture2d管道的绘制可以使用某些测试颜色.所以问题是，为什么SurfaceTexture不能将其表面转换为纹理?

Update 4: Based on the pipeline in update 1 (surface->external texture via surface texture -> fbo -> texture 2d) I know the SurfaceTexture isn't properly converting its surface to a texture. I can get correctly drawn pictures from its surface via pixelcopy and I can confirm my FBO drawing to texture2d pipeline works with some test colors. So the question is, why can't the SurfaceTexture convert its surface to a texture?

我用Java生成了Texture并将其指针传递回Unity:

I generate a Texture in Java and pass its pointer back to Unity:

public void initGLTexture()
{
    Log.d("Unity", "initGLTexture");
    int textures[] = new int[1];
    GLES20.glGenTextures(1, textures, 0);
    mTextureId = textures[0];

    GLES20.glBindTexture(GL_TEXTURE_EXTERNAL_OES, mTextureId);
    GLES20.glTexParameterf(GL_TEXTURE_EXTERNAL_OES, GLES20.GL_TEXTURE_MIN_FILTER, GLES20.GL_LINEAR);
    GLES20.glTexParameterf(GL_TEXTURE_EXTERNAL_OES, GLES20.GL_TEXTURE_MAG_FILTER, GLES20.GL_LINEAR);
    GLES20.glTexParameteri(GL_TEXTURE_EXTERNAL_OES, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE);
    GLES20.glTexParameteri(GL_TEXTURE_EXTERNAL_OES, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE);
}

我从id(在Java中)创建SurfaceTexture:

I create a SurfaceTexture from the id (in Java):

mSurfaceTexture = new SurfaceTexture(mTextureId);
mSurfaceTexture.setDefaultBufferSize(512, 512);

我使用第三方库 GeckoView 渲染到表面纹理.我从Unity的OnRenderObject()调用以下方法，以将所有GL渲染保持在同一线程上:

I use a third-party library, GeckoView, to render onto the Surface of the SurfaceTexture. I call the following method from Unity's OnRenderObject() to keep all GL rendering on the same thread:

mSurfaceTexture.updateTexImage();

我知道上面的代码允许在表面上正确绘制.

I know the above code allows proper drawing onto the surface.

我在Unity中调用以下内容来加载纹理:

I call the following in Unity to load the texture:

_imageTexture2D = Texture2D.CreateExternalTexture(
        512,512,TextureFormat.RGBA32,false,true,(IntPtr) mTextureId);
_rawImage.texture = _imageTexture2D;

为什么RawImage带有纹理的仅显示看起来像精灵的东西，应该是网页?

Why does the RawImage with the texture applied show only this sprite-looking thing, which should be a webpage?

更新1:因此，我一直在研究以下假设:使用Gecko绘制到Surface，并使用SurfaceTexture将此表面渲染为GL_TEXTURE_EXTERNAL_OES.由于无法在Unity上显示此图像(不知道为什么)，因此我将该纹理绘制到帧缓冲区，并将帧缓冲区中的像素复制到GL_TEXTURE_2D.我在texture_2d中获得了一个网页(在带有imageview和glReadPixels的仿真器中).但是，当我将工作导入Unity以测试到目前为止管道是否还可以时，我只是得到了黑屏.我可以通过PixelCopy api获取表面图像.

Update 1: So I've been working on the hypothesis of: use Gecko to draw to the Surface, and use a SurfaceTexture to render this surface to a GL_TEXTURE_EXTERNAL_OES. Since I can't display this on Unity (not sure why) I am drawing this texture to a frame buffer and copying the pixels in the framebuffer to a GL_TEXTURE_2D. I am getting a web page in the texture_2d (in the emulator with an imageview and glReadPixels). However, when I import the work into Unity to test if the pipeline is okay thus far I just get a black screen. I CAN get images of the surface via the PixelCopy api.

这是我的FBO概述代码-我的渲染代码来自grafika的

Here is my FBO overview code - my rendering code comes from grafika's texture2D program:

    // bind display buffer
    GLES20.glBindFramebuffer(GLES20.GL_FRAMEBUFFER, mFrameBufferId);
    GlUtil.checkGlError("glbindframebuffer");

    // unbind external texture to make sure it's fresh
    GLES20.glBindTexture(GLES11Ext.GL_TEXTURE_EXTERNAL_OES, 0);
    GlUtil.checkGlError("glunbindexternaltex");

   // bind source texture (done in drawFrame as well )
    GLES20.glBindTexture(GL_TEXTURE_EXTERNAL_OES, mOffscreenTextureId);
    GlUtil.checkGlError("glBindFramebuffer");

    // draw to frame buffer
    GLES20.glClearColor(0.0f, 0.0f, 0.0f, 1.0f);    // again, only really need to
    GLES20.glClear(GLES20.GL_COLOR_BUFFER_BIT);     //  clear pixels outside rect

    mFullScreen.drawFrame(mOffscreenTextureId, mIdentityMatrix);

    // unbind source texture
    GLES20.glBindTexture(GLES20.GL_TEXTURE_2D,0);
    GlUtil.checkGlError("glBindTexture2d");

    GLES20.glBindTexture(GLES11Ext.GL_TEXTURE_EXTERNAL_OES, 0);
    GlUtil.checkGlError("glunbindexternaltex");

    // make sure we're still bound to fbo
    GLES20.glBindFramebuffer(GLES20.GL_FRAMEBUFFER, mFrameBufferId);
    GlUtil.checkGlError("glBindTexture2d");

    // copy pixels from frame buffer to display texture
     GLES20.glCopyTexImage2D(GLES20.GL_TEXTURE_2D,0,GLES20.GL_RGBA,0,0,512,512,0);

    // read pixels from the display buffer to imageview for debugging
    BitmapDisplay.mBitmap = SavePixels(0,0,512,512);

    // unbind texture
    GLES20.glBindTexture(GLES20.GL_TEXTURE_2D,0); 

这是我的播放器设置>其他:

Here's my player settings > other:

更新2:可能尝试的管道:通过

Update 2: Possible pipeline to try: call the draw function of the external texture to FBO (attached to Unity's texture_2d) in C++ via this interface.

更新3:从本机代码中调用Java函数，该函数负责通过GL.IssuePluginEvent将表面纹理从SurfaceTexture绘制到FBO并绘制到Unity的纹理，这与第一次更新中一样. .它将在模拟器中显示图像，但在Unity中不显示图像.

Update 3: Calling the Java functions from native code that are responsible for drawing the texture from the SurfaceTexture to the FBO to Unity's texture via the GL.IssuePluginEvent produce a black texture as in the first update. It will show images in the emulator but not in Unity.

推荐答案

几个月前，我不得不做类似的任务，发现正确的管道正在Unity中创建纹理，并在C和最终在Java层中对其进行更新.

I had to do a similar task a couple of months ago and found out that the correct pipeline is creating a texture in Unity, obtaining a native pointer in C and finally updating it in the Java layer.

请看一下这个示例项目，它应该给您不同的视角.

Please take a look at this sample project, it should give you a different perspective.

https://github.com/robsondepaula/unity-android-native-camera

致谢

这篇关于Android插件中的SurfaceTexture在Unity中不起作用的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！