将位图帧渲染到 Surface 进行编码 [英] Render Bitmap frames to Surface for encoding

问题描述

我的目标是接收一个 M4V 视频文件，将一段视频解码为 PNG 帧，修改这些帧，然后将修剪后的视频重新编码(也为 M4V).

My goal is to take in a M4V video file, decode a segment of the video as PNG frames, modify these frames, and re-encode the trimmed video (also to M4V).

工作流程是这样的:[Input Video] ->导出帧 ->修改框架 ->编码帧 ->[输出视频].

对于解码过程，我一直在参考 bigflake 示例.使用 ExtractMpegFramesTest 示例代码，我能够生成 .m4v 文件中的 >Bitmap 帧并将帧导出为 PNG 文件.

For the decode process, I have been referencing the bigflake examples. Using the ExtractMpegFramesTest example code I was able to generate Bitmap frames from an .m4v file and export frames as PNG files.

现在我正在尝试重新编码过程，使用 EncodeAndMuxTest 尝试为编码创建另一组类的示例.

Now I am attempting the re-encoding process, using the EncodeAndMuxTest example in attempts to create another set of classes for encoding.

我遇到的问题是，示例代码似乎在 OpenGL 中生成原始帧.我有一系列 Bitmaps 我想编码/渲染到 CodecInputSurface 对象.几乎与解码过程相反.

The issue I am running into is, the example code seems to generate raw frames in OpenGL. I have a series of Bitmaps that I want to encode/render to the CodecInputSurface object. Pretty much the reverse of what the decoding process does.

大部分示例代码都很好，看来我只需要修改generateSurfaceFrame() 将Bitmap 渲染到Surface> 使用 OpenGL.

The majority of the example code is just fine, it seems I just need to modify generateSurfaceFrame() to render the Bitmap to the Surface with OpenGL.

这是我迄今为止的代码:

Here is the code that I have thus far:

// Member variables (see EncodeAndMuxTest example)
private MediaCodec encoder;
private CodeInputSurface inputSurface;
private MediaMuxer muxer;
private int trackIndex;
private boolean hasMuxerStarted;
private MediaCodec.BufferInfo bufferInfo;

// This is called for each frame to be rendered into the video file
private void encodeFrame(Bitmap bitmap)
{
    int textureId = 0;

    try
    {
        textureId = loadTexture(bitmap);

        // render the texture here?
    }
    finally
    {
        unloadTexture(textureId);
    }
}

// Loads a texture into OpenGL
private int loadTexture(Bitmap bitmap)
{
    final int[] textures = new int[1];
    GLES20.glGenTextures(1, textures, 0);

    int textureWidth = bitmap.getWidth();
    int textureHeight = bitmap.getHeight();

    GLES20.glBindTexture(GLES20.GL_TEXTURE_2D, textures[0]);
    GLUtils.texImage2D(GLES20.GL_TEXTURE_2D, 0, bitmap, 0);

    GLES20.glTexParameteri(GLES20.GL_TEXTURE_2D,
            GLES20.GL_TEXTURE_MIN_FILTER, GLES20.GL_LINEAR);
    GLES20.glTexParameteri(GLES20.GL_TEXTURE_2D,
            GLES20.GL_TEXTURE_MAG_FILTER, GLES20.GL_NEAREST);
    GLES20.glTexParameteri(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_WRAP_S,
            GLES20.GL_CLAMP_TO_EDGE);
    GLES20.glTexParameteri(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_WRAP_T,
            GLES20.GL_CLAMP_TO_EDGE);

    return textures[0];
}

// Unloads a texture from OpenGL
private void unloadTexture(int textureId)
{
    final int[] textures = new int[1];
    textures[0] = textureId;

    GLES20.glDeleteTextures(1, textures, 0);
}

我觉得我应该能够使用 ExtractMpegFramesTest 示例中的 STextureRender 来实现类似的效果，但它只是不适合我.

I feel like I should be able to use the STextureRender from the ExtractMpegFramesTest example to achieve similar, but it's just not clicking for me.

另一件事是性能，我真的在努力获得有效的编码.我将编码 90-450 帧视频(3-15 秒 @ 30fps)，所以这应该只需要几秒钟.

Another thing is performance, which I really am trying to get efficient encoding. I will be encoding 90-450 frames of video (3-15 seconds @ 30fps), so this should only take several seconds hopefully.

推荐答案

我能够将位图帧渲染到表面进行编码.我使用 MediaCodec + MediaMuxer 使用 InputSurface 对位图帧进行编码，并使用 OpenGL 渲染位图.

I was able to render bitmap frames to a surface for encoding. I used MediaCodec + MediaMuxer to encode bitmap frames using the InputSurface and rendering the bitmaps using OpenGL.

看起来你缺少的是打开 gl 命令来渲染纹理

Looks like all you are missing is open gl commands to render the texture

为了解决这个问题，我还添加了一些额外的 open gl 命令来使用顶点着色器渲染纹理.

To fix this issue I also added some additional open gl commands to render the texture using a vertex shader.

有关使用 open gl 渲染纹理的更多详细信息，请参阅此项目和 util 类 https://github.com/rsri/Pic2Fro/blob/b4fe69b44343dab2515c3fd6e769f3370bf31312/app/src/main/java/com/Util/pic2fro/.java

See this project and util class for more details on rendering a texture using open gl https://github.com/rsri/Pic2Fro/blob/b4fe69b44343dab2515c3fd6e769f3370bf31312/app/src/main/java/com/pic2fro/pic2fro/util/Util.java

在代码段中的 GLUtils.texImage2D(..) 之后使用纹理句柄和适当的宽度和高度调用 renderTexture(..) 将解决所有位图渲染问题.

Calling the renderTexture(..) with the texture handle and appropriate width and height after GLUtils.texImage2D(..) in your snippet will fix all bitmap rendering issues.

另请参阅我的相关答案herehttps://stackoverflow.com/a/49331295/7602598

See also my related answer here https://stackoverflow.com/a/49331295/7602598

这篇关于将位图帧渲染到 Surface 进行编码的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！