android ffmpeg opengl es渲染电影 [英] android ffmpeg opengl es render movie

问题描述

我正在尝试通过NDK渲染视频，添加一些仅在sdk中不支持的功能。我正在使用FFmpeg解码视频，并可以通过ndk进行编译，并使用这个作为起点。我已经修改了这个例子，而不是使用glDrawTexiOES绘制纹理，我已经设置了一些顶点，并且渲染纹理（opengl es的渲染四边形）。

I am trying to render video via the NDK, to add some features that just aren't supported in the sdk. I am using FFmpeg to decode the video and can compile that via the ndk, and used this as a starting point. I have modified that example and instead of using glDrawTexiOES to draw the texture I have setup some vertices and am rendering the texture on top of that (opengl es way of rendering quad).

下面是我正在做的渲染，但创建glTexImage2D很慢。我想知道是否有任何方法加快速度，或者给出加速的速度，如尝试在后台设置一些纹理并渲染预设置的纹理。或者如果还有其他方法可以更快地将视频画面绘制到Android屏幕中？目前我只能得到约12fps。

Below is what I am doing to render, but creating the glTexImage2D is slow. I want to know if there is any way to speed this up, or give the appearance of speeding this up, such as trying to setup some textures in the background and render pre-setup textures. Or if there is any other way to more quickly draw the video frames to screen in android? Currently I can only get about 12fps.

glClear(GL_COLOR_BUFFER_BIT);
glEnableClientState(GL_VERTEX_ARRAY);
glEnableClientState(GL_TEXTURE_COORD_ARRAY);
glBindTexture(GL_TEXTURE_2D, textureConverted);

//this is slow
glTexImage2D(GL_TEXTURE_2D, /* target */
0, /* level */
GL_RGBA, /* internal format */
textureWidth, /* width */
textureHeight, /* height */
0, /* border */
GL_RGBA, /* format */
GL_UNSIGNED_BYTE,/* type */
pFrameConverted->data[0]);

glEnableClientState(GL_TEXTURE_COORD_ARRAY);
glTexCoordPointer(2, GL_FLOAT, 0, texCoords);
glVertexPointer(3, GL_FLOAT, 0, vertices);
glDrawElements(GL_TRIANGLES, 6, GL_UNSIGNED_BYTE, indices);
glDisableClientState(GL_VERTEX_ARRAY);
glDisableClientState(GL_TEXTURE_COORD_ARRAY);

编辑
我更改了我的代码，以初始化一个gltextImage2D一次，并用glSubTexImage2D修改它，它对帧速率没有太大的改善。

EDIT I changed my code to initialize a gltextImage2D only once, and modify it with glSubTexImage2D, it didn't make much of an improvement to the framerate.

然后，我修改了修改NDK上的本机Bitmap对象的代码。使用这种方法，我有一个后台线程运行，处理下一个帧并填充本地的位图对象。我认为这有潜力，但是我需要将AVFrame对象从FFmpeg转换为本地位图。下面是我正在用的是转换，一种暴力的做法。有没有办法提高速度或优化这种转换？

I then modified the code to modify a native Bitmap object on the NDK. With this approach I have a background thread that runs that process the next frames and populates the bitmap object on the native side. I think this has potential, but I need to get the speed increased of converting the AVFrame object from FFmpeg into a native bitmap. Below is currently what I am using to convert, a brute force approach. Is there any way to increase the speed of this or optimize this conversion?

static void fill_bitmap(AndroidBitmapInfo*  info, void *pixels, AVFrame *pFrame)
{
uint8_t *frameLine;

int  yy;
for (yy = 0; yy < info->height; yy++) {
    uint8_t*  line = (uint8_t*)pixels;
    frameLine = (uint8_t *)pFrame->data[0] + (yy * pFrame->linesize[0]);

    int xx;
    for (xx = 0; xx < info->width; xx++) {
        int out_offset = xx * 4;
        int in_offset = xx * 3;

        line[out_offset] = frameLine[in_offset];
        line[out_offset+1] = frameLine[in_offset+1];
        line[out_offset+2] = frameLine[in_offset+2];
        line[out_offset+3] = 0;
    }
    pixels = (char*)pixels + info->stride;
}
}

推荐答案

是，纹理（和缓冲区，着色器和帧缓冲区）的创建速度很慢。

Yes, texture (and buffer, and shader, and framebuffer) creation is slow.

这就是为什么你应该只创建纹理一次。创建后，您可以通过调用 glSubTexImage2D 。

That's why you should create texture only once. After it is created, you can modify its data by calling glSubTexImage2D.

并且更快地上传纹理数据 - 创建两个纹理。当您使用一个显示时，将纹理数据从ffmpeg上传到第二个。当您显示第二个数据时，将数据上传到第一个。从头开始重复。

And to make uploading texture data more faster - create two textures. While you use one to display, upload texture data from ffmpeg to second one. When you display second one, upload data to first one. And repeat from beginning.

我觉得还不是很快。您可以尝试使用允许从NDK访问Bitmap对象像素的jnigraphics库。之后 - 你只是在java边屏幕上显示Bitmap。

I think it will still be not very fast. You could try to use jnigraphics library that allows to access Bitmap object pixels from NDK. After that - you just diplay this Bitmap on screen on java side.

这篇关于android ffmpeg opengl es渲染电影的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！