将位图帧渲染到 Surface 以进行编码 [英] Render Bitmap frames to Surface for encoding
问题描述
我的目标是获取一个 M4V 视频文件,将视频片段解码为 PNG 帧,修改这些帧,然后重新编码修剪后的视频(也为 M4V).
My goal is to take in a M4V video file, decode a segment of the video as PNG frames, modify these frames, and re-encode the trimmed video (also to M4V).
工作流程是这样的:[Input Video] ->导出框架 ->修改框架->编码帧->[输出视频]
.
对于解码过程,我一直在参考 bigflake 示例.使用 ExtractMpegFramesTest 示例代码,我能够生成 Bitmap
来自 .m4v
文件的帧并将帧导出为 PNG 文件.
For the decode process, I have been referencing the bigflake examples. Using the ExtractMpegFramesTest example code I was able to generate Bitmap
frames from an .m4v
file and export frames as PNG files.
现在我正在尝试重新编码过程,使用 EncodeAndMuxTest 尝试创建另一组用于编码的类的示例.
Now I am attempting the re-encoding process, using the EncodeAndMuxTest example in attempts to create another set of classes for encoding.
我遇到的问题是,示例代码似乎在 OpenGL 中生成原始帧.我有一系列 Bitmaps
我想编码/渲染到 CodecInputSurface
对象.几乎与解码过程相反.
The issue I am running into is, the example code seems to generate raw frames in OpenGL. I have a series of Bitmaps
that I want to encode/render to the CodecInputSurface
object. Pretty much the reverse of what the decoding process does.
大部分示例代码都很好,看来我只需要修改 generateSurfaceFrame()
以将 Bitmap
渲染到 Surface
使用 OpenGL.
The majority of the example code is just fine, it seems I just need to modify generateSurfaceFrame()
to render the Bitmap
to the Surface
with OpenGL.
这是我到目前为止的代码:
Here is the code that I have thus far:
// Member variables (see EncodeAndMuxTest example)
private MediaCodec encoder;
private CodeInputSurface inputSurface;
private MediaMuxer muxer;
private int trackIndex;
private boolean hasMuxerStarted;
private MediaCodec.BufferInfo bufferInfo;
// This is called for each frame to be rendered into the video file
private void encodeFrame(Bitmap bitmap)
{
int textureId = 0;
try
{
textureId = loadTexture(bitmap);
// render the texture here?
}
finally
{
unloadTexture(textureId);
}
}
// Loads a texture into OpenGL
private int loadTexture(Bitmap bitmap)
{
final int[] textures = new int[1];
GLES20.glGenTextures(1, textures, 0);
int textureWidth = bitmap.getWidth();
int textureHeight = bitmap.getHeight();
GLES20.glBindTexture(GLES20.GL_TEXTURE_2D, textures[0]);
GLUtils.texImage2D(GLES20.GL_TEXTURE_2D, 0, bitmap, 0);
GLES20.glTexParameteri(GLES20.GL_TEXTURE_2D,
GLES20.GL_TEXTURE_MIN_FILTER, GLES20.GL_LINEAR);
GLES20.glTexParameteri(GLES20.GL_TEXTURE_2D,
GLES20.GL_TEXTURE_MAG_FILTER, GLES20.GL_NEAREST);
GLES20.glTexParameteri(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_WRAP_S,
GLES20.GL_CLAMP_TO_EDGE);
GLES20.glTexParameteri(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_WRAP_T,
GLES20.GL_CLAMP_TO_EDGE);
return textures[0];
}
// Unloads a texture from OpenGL
private void unloadTexture(int textureId)
{
final int[] textures = new int[1];
textures[0] = textureId;
GLES20.glDeleteTextures(1, textures, 0);
}
我觉得我应该能够使用 ExtractMpegFramesTest 示例中的 STextureRender
来实现类似的效果,但它只是不适合我.
I feel like I should be able to use the STextureRender
from the ExtractMpegFramesTest example to achieve similar, but it's just not clicking for me.
另一件事是性能,我真的在努力获得高效的编码.我将编码 90-450 帧的视频(3-15 秒 @ 30fps),所以这应该只需要几秒钟.
Another thing is performance, which I really am trying to get efficient encoding. I will be encoding 90-450 frames of video (3-15 seconds @ 30fps), so this should only take several seconds hopefully.
推荐答案
我能够将位图帧渲染到表面进行编码.我使用 MediaCodec + MediaMuxer 使用 InputSurface 对位图帧进行编码,并使用 OpenGL 渲染位图.
I was able to render bitmap frames to a surface for encoding. I used MediaCodec + MediaMuxer to encode bitmap frames using the InputSurface and rendering the bitmaps using OpenGL.
看起来你所缺少的只是打开 gl 命令来渲染纹理
Looks like all you are missing is open gl commands to render the texture
为了解决这个问题,我还添加了一些额外的 open gl 命令来使用顶点着色器渲染纹理.
To fix this issue I also added some additional open gl commands to render the texture using a vertex shader.
See this project and util class for more details on rendering a texture using open gl https://github.com/rsri/Pic2Fro/blob/b4fe69b44343dab2515c3fd6e769f3370bf31312/app/src/main/java/com/pic2fro/pic2fro/util/Util.java
在代码段中的 GLUtils.texImage2D(..) 之后使用纹理句柄和适当的宽度和高度调用 renderTexture(..) 将修复所有位图渲染问题.
Calling the renderTexture(..) with the texture handle and appropriate width and height after GLUtils.texImage2D(..) in your snippet will fix all bitmap rendering issues.
也可以在这里查看我的相关答案https://stackoverflow.com/a/49331295/7602598
See also my related answer here https://stackoverflow.com/a/49331295/7602598
这篇关于将位图帧渲染到 Surface 以进行编码的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!