QVideoFrame::map() 崩溃 [英] QVideoFrame::map() crashes
问题描述
我正在尝试使用 QCamera 从相机获取图像.我想我应该从 QAbstractVideoSurface 派生并实现 present()，其中一个 QVideoFrame，代表相机捕获的当前图像，被发送为一个参数.
I am trying to get images from the camera using QCamera.
I thought I should derive from QAbstractVideoSurface and implement present(), where a QVideoFrame, representing the current image captured by the camera, is sent as a parameter.
因为我需要做一些处理，我尝试map()我的框架，用bits()获取数据，做我必须做的事，然后unmap() 它.但是我在 map()
As I need to do some processing, I tried to map() my frame, get the data with bits(), do whatever I have to do, then unmap() it. However I have a crash on map()
这是我得到的错误:
W libTest.so: (null):0 ((null)): Unsupported viewfinder pixel format
D SensorManager: registerListener :: 6, LSM6DSL Acceleration Sensor, 200000, 0,
E libEGL : call to OpenGL ES API with no current context (logged once per thread)
E GLConsumer: [SurfaceTexture-0-546-0] attachToContext: invalid current EGLDisplay
F libc : Fatal signal 11 (SIGSEGV), code 1, fault addr 0x4 in tid 663 (qtMainLoopThrea)
我做错了什么?
这是一个完整的应用程序代码:
Here is a full application code :
///////////////////////////////////////////////
//main.cpp
///////////////////////////////////////////////
#include <QGuiApplication>
#include <QQmlApplicationEngine>
#include <QQmlContext>
#include "camera_engine.h"
int main(int argc, char *argv[])
{
QCoreApplication::setAttribute(Qt::AA_EnableHighDpiScaling);
QGuiApplication app(argc, argv);
QQmlApplicationEngine engine;
CameraEngine camEngine;
engine.rootContext()->setContextProperty("cameraEngine", &camEngine);
engine.load(QUrl(QStringLiteral("qrc:/main.qml")));
if (engine.rootObjects().isEmpty())
return -1;
return app.exec();
}
///////////////////////////////////////////////
//camera_engine.h
///////////////////////////////////////////////
#ifndef __CAMERA_ENGINE_H__
#define __CAMERA_ENGINE_H__
#include <QCamera>
#include "image_reader.h"
class CameraEngine : public QObject {
Q_OBJECT
public:
explicit CameraEngine(QCamera::Position pos = QCamera::BackFace);
~CameraEngine();
public slots:
void start();
private:
QCamera mCamera;
ImageReader mImageReader;
};
#endif // __CAMERA_ENGINE_H__
///////////////////////////////////////////////
//camera_engine.cpp
///////////////////////////////////////////////
#include "camera_engine.h"
CameraEngine::CameraEngine(QCamera::Position pos) : mCamera(pos)
{
mCamera.setViewfinder(&mImageReader);
QCameraViewfinderSettings viewFinderSettings;
viewFinderSettings.setResolution(640, 480);
viewFinderSettings.setMinimumFrameRate(30);
viewFinderSettings.setMaximumFrameRate(30);
viewFinderSettings.setPixelFormat(QVideoFrame::Format_RGB24);
mCamera.setViewfinderSettings(viewFinderSettings);
}
CameraEngine::~CameraEngine()
{
if (mCamera.state() == QCamera::ActiveState) {
mCamera.stop();
}
}
void CameraEngine::start()
{
mCamera.start();
}
///////////////////////////////////////////////
//image_reader.h
///////////////////////////////////////////////
#ifndef CAMERA_IMAGE_READER_H
#define CAMERA_IMAGE_READER_H
#include <QAbstractVideoSurface>
class ImageReader : public QAbstractVideoSurface {
Q_OBJECT
public:
ImageReader() = default;
~ImageReader() = default;
virtual bool present(const QVideoFrame& frame);
virtual QList<QVideoFrame::PixelFormat> supportedPixelFormats(QAbstractVideoBuffer::HandleType type) const;
};
#endif // CAMERA_IMAGE_READER_H
///////////////////////////////////////////////
//image_reader.cpp
///////////////////////////////////////////////
#include "image_reader.h"
#include <QDebug>
bool ImageReader::present(const QVideoFrame &frame)
{
QVideoFrame currentFrame = frame;
currentFrame.map(QAbstractVideoBuffer::ReadOnly); //crashes here
// Do something
currentFrame.unmap();
return true;
}
QList<QVideoFrame::PixelFormat> ImageReader::supportedPixelFormats(QAbstractVideoBuffer::HandleType type) const
{
Q_UNUSED(type)
return QList<QVideoFrame::PixelFormat>() << QVideoFrame::Format_RGB24;
}
///////////////////////////////////////////////
//main.qml
///////////////////////////////////////////////
import QtQuick 2.11
import QtQuick.Controls 2.2
ApplicationWindow {
id: window
visible: true
width: 640
height: 480
Component.onCompleted: cameraEngine.start()
}
所以，我认为这可能是因为我的 QVideoFrame 存储为 OpenGL 纹理，而我的 present() 函数可能未在 OpenGL 线程上运行，因此找不到 OpenGL ES语境.有没有办法确保它在正确的线程上运行?
So, I think this might be because my QVideoFrame is stored as an OpenGL texture, and my present() function might not be running on the OpenGL thread, thus not finding the OpenGL ES Context.
Is there a way to make sure it's running on the right thread ?
Edit2:我发现了这个:http://doc.qt.io/qt-5/qtquick-scenegraph-openglunderqml-example.html
I found this : http://doc.qt.io/qt-5/qtquick-scenegraph-openglunderqml-example.html
也许我可以修改此代码以获得有效的 OpenGL ES 上下文.不幸的是，我现在没有时间去做.我会在那个星期一尝试，除非有人有更好的建议，然后告诉你结果.
Maybe I can adapt this code to get a valid OpenGL ES context. Unfortunately I have no time to do it now. I'll try that Monday, unless someone have a better suggestion, and tell you the results.
Edit3:所以，显然我的解决方案不是很好，我在 initializeOpenGLFunctions();
So, apparently my solution was not the good one, I get a crash (SIGSEGV) on initializeOpenGLFunctions();
我看到了 Antonio Dias 的回答，使用带有 grabToImage 功能的 VideoOutput，尝试了一下，它似乎有效，但是，如果我理解正确，grabToImage 在 CPU 内存中绘制"VideoOutput，但在此过程中我丢失了我计划使用 QMediaMetaData 获取的一些元数据.
I saw Antonio Dias' answer, using a VideoOutput with the function grabToImage, tried it, and it seemed to work, but, if I understand correctly, grabToImage "draws" the VideoOutput in CPU memory, and I lose some metadata I was planning to get with QMediaMetaData in the process.
我也试过直接用NDK，但是摄像头要求API级别至少24，而且即使设置了我找到的所有相关设置，它似乎也没有使用它.
I also tried to use the NDK directly, but the camera requires an API level of at least 24, and even after setting all relevant settings I found, it does not seem to use it.
Edit4:我实际上不知道我做了什么，但我的程序最终使用了正确的 API 级别，所以我现在将使用 NDK 解决方案.
I actually do not know what I did, but my program ended up using the right API level, so I'll go with the NDK solution for now.
推荐答案
我终于设法让它发挥作用.显然 QCamera 在 Android 上有一些问题，所以我不得不使用 QML Camera，并用 VideoOutput 显示它，并通过应用过滤器我可以得到我的图像.
I finally managed to make it work. Apparently QCamera has some problems on Android, so I had to use a QML Camera, and display it with a VideoOutput, and apply a filter through which I can get my images.
此过滤器由两部分组成:一部分源自 QAbstractVideoFilter，另一个源自QVideoFilterRunnable.
This filter is made in two parts : one deriving from QAbstractVideoFilter, the other deriving from QVideoFilterRunnable.
代码如下:
////////////////////////////////////////////////////////////////////
// main.cpp
////////////////////////////////////////////////////////////////////
#include <QGuiApplication>
#include <QQmlApplicationEngine>
#include "myfilter.hpp"
int main(int argc, char *argv[])
{
QCoreApplication::setAttribute(Qt::AA_EnableHighDpiScaling);
QGuiApplication app(argc, argv);
QQmlApplicationEngine engine;
qmlRegisterType<MyFilter>("example.myfilter", 1, 0, "MyFilter");
engine.load(QUrl(QStringLiteral("qrc:/main.qml")));
if (engine.rootObjects().isEmpty())
return -1;
return app.exec();
}
////////////////////////////////////////////////////////////////////
// myfilter.hpp
////////////////////////////////////////////////////////////////////
#ifndef MYFILTER_H
#define MYFILTER_H
#include <QAbstractVideoFilter>
class MyFilterRunnable : public QVideoFilterRunnable {
public:
QVideoFrame run(QVideoFrame *input, const QVideoSurfaceFormat &surfaceFormat, RunFlags flags);
};
class MyFilter : public QAbstractVideoFilter
{
public:
QVideoFilterRunnable* createFilterRunnable();
};
#endif // MYFILTER_H
////////////////////////////////////////////////////////////////////
// myfilter.cpp
////////////////////////////////////////////////////////////////////
#include "myfilter.hpp"
#include <QOpenGLContext>
#include <QOpenGLFunctions>
QVideoFrame MyFilterRunnable::run(QVideoFrame *input, const QVideoSurfaceFormat &surfaceFormat, QVideoFilterRunnable::RunFlags flags)
{
QImage img(input->width(), input->height(), QImage::Format_RGBA8888);
bool success = false;
if (input->handleType() == QAbstractVideoBuffer::GLTextureHandle) {
GLuint textureId = input->handle().toUInt();
QOpenGLContext *ctx = QOpenGLContext::currentContext();
QOpenGLFunctions *f = ctx->functions();
GLuint fbo;
f->glGenFramebuffers(1, &fbo);
GLuint prevFbo;
f->glGetIntegerv(GL_FRAMEBUFFER_BINDING, (GLint *) &prevFbo);
f->glBindFramebuffer(GL_FRAMEBUFFER, fbo);
f->glFramebufferTexture2D(GL_FRAMEBUFFER, GL_COLOR_ATTACHMENT0, GL_TEXTURE_2D, textureId, 0);
f->glReadPixels(0, 0, input->width(), input->height(), GL_RGBA, GL_UNSIGNED_BYTE, img.bits());
f->glBindFramebuffer(GL_FRAMEBUFFER, prevFbo);
success = true;
} // else handle other types
if( success ) {
// Process image
return QVideoFrame(img);
} else {
return *input; //Could not apply filter, return unmodified input
}
}
QVideoFilterRunnable *MyFilter::createFilterRunnable()
{
return new MyFilterRunnable;
}
////////////////////////////////////////////////////////////////////
// main.qml
////////////////////////////////////////////////////////////////////
import QtQuick 2.11
import QtQuick.Controls 2.2
import QtMultimedia 5.9
import example.myfilter 1.0
ApplicationWindow {
id: window
visible: true
width: 640
height: 480
Camera {
id: camera
}
MyFilter {
id: filter
}
VideoOutput {
source: camera
autoOrientation: true
filters: [ filter ]
anchors.fill: parent
}
}
(我的 run 实现改编自这里:http://code.qt.io/cgit/qt/qtmultimedia.git/tree/examples/multimedia/video/qmlvideofilter_opencl/rgbframehelper.h)
(My run implementation is adapted from here : http://code.qt.io/cgit/qt/qtmultimedia.git/tree/examples/multimedia/video/qmlvideofilter_opencl/rgbframehelper.h)
这篇关于QVideoFrame::map() 崩溃的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
