通过片段着色器将 YUV 转换为 RGB [英] YUV to RGB conversion by fragment shader

问题描述

我在将 Android 中的相机预览从 YUV 格式转换为 RGB 时遇到问题.转换的目的是应用一些效果.我尝试通过片段着色器进行转换，因为本机代码的转换速度很慢(大约 14fps).我使用的参考是 http://jyrom.tistory.com/m/post/view/id/187.我尝试将此代码移植到 Android 平台，但结果是黑绿色矩形.但是，我可以通过我得到的输出观看某种形式.你能不能帮我解决这个问题.我相信这是一个流行的问题:将效果应用于相机预览.我还提供了我的项目的测试链接:https://dl.dropbox.com/u/12829395/application/FilterGL/FilterGL.zip.谢谢你.
更新:
这是我的 onPreviewFrame 方法:

I've a problem with convertion of camera preview in Android from YUV format to RGB. The purpose of conversion is to apply some effects. I try to convert by fragment shader because convertion by native code is slow (about 14fps). The reference which I've used is http://jyrom.tistory.com/m/post/view/id/187. I try to port this code to Android platform, but the result is black-green rectangles. But, I can watch some form through the output which I get. Could you please try to help me to resolve this issue. I believe this is popular problem: apply effects to camera preview. I also give a link to my project for testing: https://dl.dropbox.com/u/12829395/application/FilterGL/FilterGL.zip. Thank you.
UPDATED:
This is my onPreviewFrame method:

public void onPreviewFrame(byte[] data, Camera camera) {
    yBuffer.put(data);
    yBuffer.position(0);

    System.arraycopy(data, U_INDEX, uData, 0, LENGTH_4 * 2);
    uBuffer.put(uData);
    uBuffer.position(0);

    System.arraycopy(data, V_INDEX, vData, 0, LENGTH_4);
    vBuffer.put(vData);
    vBuffer.position(0);
}

这是我在 onDrawFrame 方法中将字节数组绑定到 OpenGL 纹理的方式:

This is how I bind byte arrays to OpenGL texture in onDrawFrame method:

    GLES20.glUniform1i(yTexture, 1);
    GLES20.glTexImage2D(   GLES20.GL_TEXTURE_2D, 0, GLES20.GL_LUMINANCE,
            320, 240, 0, GLES20.GL_LUMINANCE, GLES20.GL_UNSIGNED_BYTE, yBuffer);
    GLES20.glTexParameteri(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_MIN_FILTER, GLES20.GL_LINEAR);
    GLES20.glTexParameteri(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_MAG_FILTER, GLES20.GL_LINEAR);

    GLES20.glUniform1i(uTexture, 2);
    GLES20.glTexImage2D(   GLES20.GL_TEXTURE_2D, 0, GLES20.GL_LUMINANCE,
            160, 120, 0, GLES20.GL_LUMINANCE, GLES20.GL_UNSIGNED_BYTE, uBuffer);
    GLES20.glTexParameteri(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_MIN_FILTER, GLES20.GL_LINEAR);
    GLES20.glTexParameteri(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_MAG_FILTER, GLES20.GL_LINEAR);

    GLES20.glUniform1i(vTexture, 3);
    GLES20.glTexImage2D(   GLES20.GL_TEXTURE_2D, 0, GLES20.GL_LUMINANCE,
            160, 120, 0, GLES20.GL_LUMINANCE, GLES20.GL_UNSIGNED_BYTE, vBuffer);
    GLES20.glTexParameteri(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_MIN_FILTER, GLES20.GL_LINEAR);
    GLES20.glTexParameteri(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_MAG_FILTER, GLES20.GL_LINEAR);

这是我的片段着色器代码:

And this is my fragment shader code:

#ifdef GL_ES
precision highp float;
#endif

varying vec2 v_texCoord;
uniform sampler2D y_texture;
uniform sampler2D u_texture;
uniform sampler2D v_texture;

void main()
{   
    float nx,ny,r,g,b,y,u,v;
    nx=v_texCoord.x;
    ny=v_texCoord.y;
    y=texture2D(y_texture,v_texCoord).r;
    u=texture2D(u_texture,v_texCoord).r;
    v=texture2D(v_texture,v_texCoord).r;

    y=1.1643*(y-0.0625);
    u=u-0.5;
    v=v-0.5;

    r=y+1.5958*v;
    g=y-0.39173*u-0.81290*v;
    b=y+2.017*u;

    gl_FragColor = vec4(r,g,b,1.0);
}

推荐答案

不确定你是否已经解决了这个问题.我的回答

Not sure if you have already fixed this problem.My answer

1. 默认情况下，相机输出为 NV12，但在片段着色器 YUV 到 RGB 中，您使用的是 YV12 -> RGB.你将不得不做setPreviewFormat(ImageFormat.YV12);，或者可以使用其他着色器

2. 有 3 种纹理，请务必这样做

1. By default Camera output is NV12, but in fragment shader YUV to RGB you are using YV12 -> RGB. You will have to do setPreviewFormat(ImageFormat.YV12);, or may be use some other shader

2. There are 3 textures , make sure you do

GLES20.glActiveTexture(GLES20.GL_TEXTURE2);GLES20.glBindTexture(GLES20.GL_TEXTURE_2D, muTextureName)

在调用任何 glTexImage2D 之前.和 glTexSubImage2D

before call to any glTexImage2D. and glTexSubImage2D

您还可以在每一帧中使用 glTexSubImage2D，并使用 glTexImage2D 一次.

You can also use glTexSubImage2D with every frame and glTexImage2D once.

U 和 V 的大小相同，至少 YV12，

size of U and V is same , atleast for YV12,

System.arraycopy(data, U_INDEX, uData, 0, LENGTH_4 * 2);

应该是System.arraycopy(data, U_INDEX, uData, 0, LENGTH_4);在代码中相应地更改大小.

should be System.arraycopy(data, U_INDEX, uData, 0, LENGTH_4); change the size accordingly in the code.

这篇关于通过片段着色器将 YUV 转换为 RGB的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！