如何将视频(帧)输入到GLSL着色器中 [英] How to input video (frames) into a GLSL shader

问题描述

我正在尝试使用GLSL进行视频处理.我正在使用OpenCV打开视频文件，并将每个帧都作为单个图像，然后要在GLSL着色器中使用每个帧

I'm trying to do video processing using GLSL. I'm using OpenCV to open a video file up and take each frame as a single image an then I want to use each frame in a GLSL shader

在GLSL中使用视频的最佳/理想/智能解决方案是什么?

What is the best/ideal/smart solution to using video with GLSL?

VideoCapture cap("movie.MOV");
Mat image;
bool success = cap.read(image);
if(!success)
{
    printf("Could not grab a frame\n\7");
    exit(0);
}

图像到纹理

 GLuint tex;
 glGenTextures(1, tex);
 glActiveTexture(GL_TEXTURE0);
 glBindTexture(GL_TEXTURE_2D, tex);
 glTexImage2D(GL_TEXTURE_2D, 0, GL_RGB, image.cols, image.rows, 0,
                GL_BGR, GL_UNSIGNED_BYTE, image.data);

 glUniform1i(glGetUniformLocation(shaderProgram, "Texture"), 0);

while循环中需要呈现什么内容?

我每次都需要重新编译/重新连接/重新链接我的着色器吗?或一旦创建并编译了我的着色器并使用glUseProgram(shaderProgram)，我可以继续向其发送新的纹理吗?

What needs to be in my render while loop?

Do I need to recompile/reattach/relink my shader every time? Or Once my shader is created and compiled and I use glUseProgram(shaderProgram) can I keep sending it new textures?

我一直用于将纹理渲染到屏幕上的当前循环如下.我该如何调整它以处理视频?我需要在哪里调用以更新着色器中使用的纹理?

The current loop I've been using to render a texture to the screen is as follows. How Could I adapt this to work with video? Where would I need to make my calls to update the texture being used in the shader?

while(!glfwWindowShouldClose(window))
{
    glClearColor(0.0f, 0.0f, 0.0f, 1.0f);
    glClear(GL_COLOR_BUFFER_BIT);

    glViewport(0,0,512,512);

    glBindFramebuffer(GL_READ_FRAMEBUFFER, frameBuffer);
    glReadBuffer(GL_COLOR_ATTACHMENT0);
    glDrawElements(GL_TRIANGLES, 6, GL_UNSIGNED_INT, 0);
    glBindFramebuffer(GL_DRAW_FRAMEBUFFER, 0);
    glBlitFramebuffer(0, 0, image.cols, image.rows, 0, 0, image.cols, image.rows, GL_COLOR_BUFFER_BIT, GL_LINEAR);


    glfwSwapBuffers(window);
    glfwPollEvents();
}

推荐答案

让我们澄清在循环之前需要发生的一些事情:

Let's clarify a few things that needs to happen before the loop:

• 通过glPixelStorei()设置像素存储模式；
• 仅使用glGenTextures()生成一个纹理，因为在循环的每次迭代中，其内容都将被新数据替换；
• 使用glCompileShader()编译着色器，使用glCreateShader()创建一个着色器对象，调用glCreateProgram()创建一个程序，调用glAttachShader()将着色器对象附加到新程序，最后将glLinkProgram()附加到让一切准备就绪.

• Set the pixel storage mode with glPixelStorei();
• Generate only one texture with glGenTextures(), because at every iteration of the loop its content will be replaced with new data;
• Compile the shader with glCompileShader(), use glCreateShader() to create a shader object, invoke glCreateProgram()to create a program, call glAttachShader() to attach the shader object to the new program, and finally glLinkProgram() to make everything ready to go.

也就是说，每次循环迭代都必须:

• 清除颜色和深度缓冲区；
• 使用恒等矩阵加载 modelview 矩阵；
• 指定绘图发生的位置glTranslatef();
• 从视频中检索新帧；
• 启用适当的纹理目标，对其进行绑定，然后使用glTexImage2D();
将帧传输到GPU
• 调用glUseProgram()以激活您的GLSL着色器；
• 使用GL_QUADS或其他方法绘制2D面；
• 使用glUseProgram(0)禁用程序；
• 使用glDisable(GL_TEXTURE_YOUR_TEXTURE_TARGET)禁用纹理目标；

• Clear the color and depth buffer;
• Load the modelview matrix with the identity matrix;
• Specify the location where the drawing is going to happen glTranslatef();
• Retrieve a new frame from the video;
• Enable the appropriate texture target, bind it and then transfer the frame to the GPU with glTexImage2D();
• Invoke glUseProgram() to activate your GLSL shader;
• Draw a 2D face using GL_QUADS or whatever;
• Disable the program with glUseProgram(0);
• Disable the texture target with glDisable(GL_TEXTURE_YOUR_TEXTURE_TARGET);

这或多或少是需要做的.

This is more or less what needs to be done.

顺便: 这是我的OpenCV/OpenGL/Qt应用程序 ，可从相机中检索帧并将其显示在窗口中.不过没有着色器.

By the way: here's my OpenCV/OpenGL/Qt application that retrieves frames from the camera and displays it in a window. No shaders, though.

祝你好运！

这篇关于如何将视频(帧)输入到GLSL着色器中的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！