具有OpenCV人脸检测功能的OpenGL字节缓冲区 [英] OpenGL byte-buffer with OpenCV face detection

问题描述

我正在尝试使用OpenCV和OpenGL在脸部贴上贴纸.

I am trying to overlay stickers on face using OpenCV and OpenGL.

我在onDrawFrame中获得了ByteBuffer:

I am getting the ByteBuffer inside the onDrawFrame:

@Override
    public void onDrawFrame(GL10 unused) {
        if (VERBOSE) {
            Log.d(TAG, "onDrawFrame tex=" + mTextureId);
        }

        mSurfaceTexture.updateTexImage();
        mSurfaceTexture.getTransformMatrix(mSTMatrix);

        byteBuffer.rewind();
        GLES20.glReadPixels(0, 0, mWidth, mHeight, GLES20.GL_RGBA, GLES20.GL_UNSIGNED_BYTE, byteBuffer);
        mat.put(0, 0, byteBuffer.array());

        if (mCascadeClassifier != null) {
            mFaces.empty();
            mCascadeClassifier.detectMultiScale(mat, mFaces);
            Log.d(TAG, "No. of faces detected : " + mFaces.toArray().length);
        }

        drawFrame(mTextureId, mSTMatrix);
}

我的mat对象使用相机预览的宽度和高度初始化:

My mat object is initialized in with camera preview width and height:

mat = new Mat(height, width, CvType.CV_8UC3);

日志返回0个面部检测.我有两个问题:

The log return 0 face detections. I have two questions:

1. 在这里我缺少使用OpenCV进行人脸检测的什么功能
2. 此外，如何提高视频帧渲染的性能/效率并进行实时人脸检测?因为 glReadPixels 需要花一些时间来执行和降低渲染速度.

1. What am I missing here for face detection using OpenCV?
2. Also, how can I improve the performance/efficiency of video frame rendering and do the realtime face detection? because glReadPixels takes time to execute and slow down the rendering.

推荐答案

在渲染任何内容之前，您正在GLES帧缓冲区上调用glReadPixels().如果希望回读SurfaceTexture渲染，则需要在drawFrame()之后执行.您可能需要考虑在屏幕外将纹理渲染到pbuffer EGLSurface，然后从中读取.

You are calling glReadPixels() on the GLES frame buffer before you've rendered anything. You'd need to do it after drawFrame() if you were hoping to read back the SurfaceTexture rendering. You may want to consider rendering the texture offscreen to a pbuffer EGLSurface instead, and reading back from that.

有几种不同的方法可以从相机获取像素数据:

There are a few different ways to get the pixel data from the Camera:

1. 使用Camera byte [] API.通常涉及软件副本，因此往往会很慢.
2. 将输出发送到 ImageReader .这使您可以立即访问原始YUV数据.
3. 将输出发送到SurfaceTexture，渲染纹理，并使用glReadPixels()读取RGB数据(这是我认为您要尝试的操作).通常这是非常快的，但是在某些设备和Android版本上，它可能会很慢.

1. Use the Camera byte[] APIs. Generally involves a software copy, so it tends to be slow.
2. Send the output to an ImageReader. This gives you immediate access to the raw YUV data.
3. Send the output to a SurfaceTexture, render the texture, read RGB data out with glReadPixels() (which is what I believe you are trying to do). This is generally very fast, but on some devices and versions of Android it can be slow.

这篇关于具有OpenCV人脸检测功能的OpenGL字节缓冲区的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！