使用ffmpeg的捕捉从微观的摄像头和音频帧并保存到文件 [英] Using ffmpeg to capture frames from webcam and audio from micro and saving to file

查看：3049
本文介绍了使用ffmpeg的捕捉从微观的摄像头和音频帧并保存到文件的处理方法，对大家解决问题具有一定的参考价值，需要的朋友们下面随着小编来一起学习吧！

问题描述

在过去的几个星期我一直在挣扎与ffmpeg的API，因为我无法找到一个明确的文件，我也很难搜索所有我找到的解决方案涉及网上不是C API，但ffmpeg.c命令行程序。我创建一个计划，需要捕捉来自一个摄像头和音频视频，在屏幕上显示的帧和两个音频和画面记录的视频文件。我还使用QT作为该项目的框架。

For the past few weeks I've been struggling with the ffmpeg API since I can not find a clear documentation and I also find it hard to search as all the solutions I find online involve not the c API but the ffmpeg.c command line program. I am creating a program which needs to capture video from a webcam and audio, show the frames on screen and record both the audio and frames to a video file. I am also using QT as a framework for this project.

我已经能够在屏幕上显示的帧，甚至记录下来，但我的问题是音频和视频两者的记录。我决定创建一个测试程序比较简单，只保存流文件而不显示在屏幕上的图像，从的 remuxing.c例如的FFmpeg的文件上。我的code是如下：

I've been able to show the frames on the screen and even record them, but my problem is the record of both the audio and video. I've decided to create a simpler program for tests, that only saves the stream to a file without showing the frames on screen, starting from the remuxing.c example on the ffmpeg documentation. My code is as follows:

//This is the variables on the .h
AVOutputFormat *ofmt;
AVFormatContext *ifmt_ctx, *ofmt_ctx;

QString cDeviceName;
QString aDeviceName;

int audioStream, videoStream;
bool done;

//The .cpp
#include "cameratest.h"
#include <QtConcurrent/QtConcurrent>
#include <QDebug>

CameraTest::CameraTest(QString cDeviceName, QString aDeviceName, QObject *parent) :
    QObject(parent)
{
    done = false;
    this->cDeviceName = cDeviceName;
    this->aDeviceName = aDeviceName;
    av_register_all();
    avdevice_register_all();
}

void CameraTest::toggleDone() {
    done = !done;
}

int CameraTest::init() {
    ofmt = NULL;
    ifmt_ctx = NULL;
    ofmt_ctx = NULL;

    QString fullDName = cDeviceName.prepend("video=") + ":" + aDeviceName.prepend("audio="); 
    qDebug() << fullDName;
    AVInputFormat *fmt = av_find_input_format("dshow");

    int ret, i;

    if (avformat_open_input(&ifmt_ctx, fullDName.toUtf8().data(), fmt, NULL) < 0) {
       fprintf(stderr, "Could not open input file '%s'", fullDName.toUtf8().data());
       return -1;
    }
    if ((ret = avformat_find_stream_info(ifmt_ctx, 0)) < 0) {
       fprintf(stderr, "Failed to retrieve input stream information");
       return -1;
    }
    av_dump_format(ifmt_ctx, 0, fullDName.toUtf8().data(), 0);
    avformat_alloc_output_context2(&ofmt_ctx, NULL, NULL, "test.avi");
    if (!ofmt_ctx) {
       fprintf(stderr, "Could not create output context\n");
       ret = AVERROR_UNKNOWN;
       return -1;
    }
    ofmt = ofmt_ctx->oformat;

    for (i = 0; i < ifmt_ctx->nb_streams; i++) {
       AVStream *in_stream = ifmt_ctx->streams[i];
       AVStream *out_stream = avformat_new_stream(ofmt_ctx, in_stream->codec->codec);

       if (ifmt_ctx->streams[i]->codec->codec_type == AVMEDIA_TYPE_VIDEO) {
           videoStream = i;
       }
       else if (ifmt_ctx->streams[i]->codec->codec_type == AVMEDIA_TYPE_AUDIO) {
           audioStream = i;
       }

       if (!out_stream) {
           fprintf(stderr, "Failed allocating output stream\n");
           ret = AVERROR_UNKNOWN;
           return -1;
       }
       ret = avcodec_copy_context(out_stream->codec, in_stream->codec);
       if (ret < 0) {
           fprintf(stderr, "Failed to copy context from input to output stream codec context\n");
           return -1;
       }
       out_stream->codec->codec_tag = 0;
       if (ofmt_ctx->oformat->flags & AVFMT_GLOBALHEADER)
           out_stream->codec->flags |= CODEC_FLAG_GLOBAL_HEADER;
    }
    av_dump_format(ofmt_ctx, 0, "test.avi", 1);
    if (!(ofmt->flags & AVFMT_NOFILE)) {
       ret = avio_open(&ofmt_ctx->pb, "test.avi", AVIO_FLAG_WRITE);
       if (ret < 0) {
           fprintf(stderr, "Could not open output file '%s'", "test.avi");
           return -1;
       }
    }
    ret = avformat_write_header(ofmt_ctx, NULL);
    if (ret < 0) {
       fprintf(stderr, "Error occurred when opening output file\n");
       return -1;
    }
    QtConcurrent::run(this, &CameraTest::grabFrames);
    return 0;
}

void CameraTest::grabFrames() {
    AVPacket pkt;
    int ret;
    while (av_read_frame(ifmt_ctx, &pkt) >= 0) {
        AVStream *in_stream, *out_stream;
        in_stream  = ifmt_ctx->streams[pkt.stream_index];
        out_stream = ofmt_ctx->streams[pkt.stream_index];
        /* copy packet */
        pkt.pts = av_rescale_q_rnd(pkt.pts, in_stream->time_base, out_stream->time_base, (AVRounding) (AV_ROUND_NEAR_INF|AV_ROUND_PASS_MINMAX));
        pkt.dts = av_rescale_q_rnd(pkt.dts, in_stream->time_base, out_stream->time_base, (AVRounding) (AV_ROUND_NEAR_INF|AV_ROUND_PASS_MINMAX));
        pkt.duration = av_rescale_q(pkt.duration, in_stream->time_base, out_stream->time_base);
        pkt.pos = -1;
        int ret = av_interleaved_write_frame(ofmt_ctx, &pkt);
        if (ret < 0) {
           qDebug() << "Error muxing packet";
           //break;
        }
        av_free_packet(&pkt);

        if(done) break;
    }
    av_write_trailer(ofmt_ctx);

    avformat_close_input(&ifmt_ctx);
    /* close output */
    if (ofmt_ctx && !(ofmt->flags & AVFMT_NOFILE))
       avio_close(ofmt_ctx->pb);
    avformat_free_context(ofmt_ctx);
    if (ret < 0 && ret != AVERROR_EOF) {
        //return -1;
       //fprintf(stderr, "Error occurred: %s\n", av_err2str(ret));
    }
}

该av_interleaved_write_frame返回一个错误的视频数据包。最终的文件只显示第一帧，但声音似乎是确定。

The av_interleaved_write_frame returns an error with the video packets. The end file shows only the first frame but the audio seems to be ok.

在控制台这是被打印的内容：

On the console this is what is printed:

Input #0, dshow, from 'video=Integrated Camera:audio=Microfone interno (Conexant 206':
  Duration: N/A, start: 146544.738000, bitrate: 1411 kb/s
    Stream #0:0: Video: rawvideo, bgr24, 640x480, 30 tbr, 10000k tbn, 30 tbc
    Stream #0:1: Audio: pcm_s16le, 44100 Hz, 2 channels, s16, 1411 kb/s
Output #0, avi, to 'test.avi':
    Stream #0:0: Video: rawvideo, bgr24, 640x480, q=2-31, 30 tbc
    Stream #0:1: Audio: pcm_s16le, 44100 Hz, 2 channels, s16, 1411 kb/s

[avi @ 0089f660] Using AVStream.codec.time_base as a timebase hint to the muxer is deprecated. Set AVStream.time_base instead.
[avi @ 0089f660] Using AVStream.codec.time_base as a timebase hint to the muxer is deprecated. Set AVStream.time_base instead.
[avi @ 0089f660] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 4396365 >= 4396365
[avi @ 0089f660] Too large number of skipped frames 4396359 > 60000
[avi @ 0089f660] Too large number of skipped frames 4396360 > 60000
[avi @ 0089f660] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 4396390 >= 4396390
[avi @ 0089f660] Too large number of skipped frames 4396361 > 60000
[avi @ 0089f660] Too large number of skipped frames 4396362 > 60000
[avi @ 0089f660] Too large number of skipped frames 4396364 > 60000
[avi @ 0089f660] Too large number of skipped frames 4396365 > 60000
[avi @ 0089f660] Too large number of skipped frames 4396366 > 60000
[avi @ 0089f660] Too large number of skipped frames 4396367 > 60000

这在我看来是一个简单的问题来解决，但我真的很大部分是无能FFmpeg的API，如果有人可能会导致我这将是伟大正确的方向！

This seems to me like a simple problem to solve but I really am mostly clueless about the ffmpeg API, if someone could lead me to the right direction that would be great!

谢谢！

推荐答案

您的问题似乎有些特定的DirectShow。不幸的是我没有获得与DirectShow的一个系统，但它看起来像捕获的症状是不是你的问题。什么是错的是混流的一部分。可以是视频包的格式不直接支持以AVI，或者可以是对包的时间戳是破

Your problem seems to be somewhat specific to DirectShow. Unfortunately I don't have access to a system with DirectShow, but from the symptom it looks like the capture is not your problem. What is wrong is the muxing part. May be the format of the video packets is not directly supported in AVI, or may be the timestamps on the packets are broken.

我会推荐几件事情，你应该尝试，一次一个：

I will recommend a few things that you should try, one at a time:


  • 尝试使用 av_write_frame 而不是 av_interleaved_write_frame 。

  • 使用更好的容器，如MP4或MKV。

  • 请不要尝试输入多路复用器数据包到一个AVI文件。在 grabFrames 取原始视频数据包，并将其转储到文件中。这应该给你一个文件，该文件是ffplay播放。 （您可能必须在指定的ffplay命令的分辨率，像素格式和格式。）

  • 做了上述结果中可播放的视频文件？如果是的话那么我建议你去code中的各个视频数据包，使用通用codeC转换色彩空间和连接code他们。 （我建议在YUV420P H264）FFmpeg的code基有两个例子，这应该是有用的 - demuxing_decoding.c 和 decoding_encoding.c 。这应该给你一个适当的视频文件。 （可播放大部分的球员。）

  • Try using av_write_frame instead of av_interleaved_write_frame.
  • Use a better container, like MP4 or MKV.
  • Do not try to mux the input packet to an avi file. In grabFrames take the raw video packets and dump them into a file. That should give you a file that is playable by ffplay. (You will probably have to specify resolution, pixel format and format in your ffplay command.)
  • Did the above result in a playable video file? If yes then I'd recommend that you decode the individual video packets, convert the colorspace and encode them using a common codec. (I recommend yuv420p in h264.) FFmpeg code base have two examples which should be useful - demuxing_decoding.c and decoding_encoding.c. That should give you a proper video file. (Playable in most players.)

我不知道DirectShow的任何东西，我不知道你的使用情况。所以，我建议重点关注FFmpeg的API。其中一些可能是矫枉过正/你想要的可能做不到。

I don't know anything about DirectShow, and I don't know your use case. So my recommendations focus on FFmpeg API. Some of it may be overkill / may not do what you want.

这篇关于使用ffmpeg的捕捉从微观的摄像头和音频帧并保存到文件的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！

查看全文
相关文章
C/C++开发最新文章
热门教程
热门工具
登录 关闭
扫码关注1秒登录
发送“验证码”获取 | 15天全站免登陆