FFMPEG API - 录制视频和音频 - 同步问题 [英] FFMPEG API - Recording video and audio - Syncing problems

问题描述

我正在开发一个能够从麦克风的网络摄像头和音频录制视频的应用程序。我一直在使用QT，但不幸的是，相机模块在Windows上不起作用，导致我使用ffmpeg录制视频/音频。

I'm developing an app which is able to record video from a webcam and audio from a microphone. I've been using QT but unfortunately the camera module does not work on windows which led me to use ffmpeg to record the video/audio.

我的相机模块正在工作除了同步的一个小问题。音频和视频有时最终会出现不同步（不到1秒钟，我会说，尽管更长的录音可能会更糟）。

My Camera module is now working well besides a slight problem with syncing. The audio and video sometimes end up out of sync by a small difference (less than 1 second I'd say, although it might be worse with longer recordings).

当我对帧进行编码时，我按以下方式添加PTS（我从 muxing.c 示例）：

When I encode the frames I add the PTS in the following way (which I took from the muxing.c example):

• 对于视频帧I逐个递增（从0开始）。


• 对于音频帧，我将PTS增加到$ code> nb_samples 音频帧（从0开始）。

• For the video frames I increment the PTS one by one (starting at 0).
• For the audio frames I increment the PTS by the nb_samples of the audio frame (starting at 0).

我以25 fps保存文件，并要求相机给我25 fps （它可以）。我也将视频帧转换为 YUV420P 格式。对于音频帧转换，我需要使用 AVAudioFifo ，因为超分辨率发送比mp4流支持更大的样本，因此我必须将它们分割成块。我使用了 transcode.c 示例。

I am saving the file at 25 fps and asking for the camera to give me 25 fps (which it can). I am also converting the video frames to the YUV420P format. For the audio frames conversion I need to use a AVAudioFifo because the microfone sends bigger samples than the mp4 stream supports, so I have to split them in chuncks. I used the transcode.c example for this.

我不知道我应该做什么来同步音频和视频。我需要使用时钟或东西来正确同步两个流？

I am out of ideas in what I should do to sync the audio and video. Do I need to use a clock or something to correctly sync up both streams?

完整的代码太大，不能在这里发布，但是如果有必要，我可以将其添加到github例如。

The full code is too big to post here but should it be necessary I can add it to github for example.

以下是编写框架的代码：

Here is the code for writing a frame:

int FFCapture::writeFrame(const AVRational *time_base, AVStream *stream, AVPacket *pkt) {
    /* rescale output packet timestamp values from codec to stream timebase */
    av_packet_rescale_ts(pkt, *time_base, stream->time_base);
    pkt->stream_index = stream->index;
    /* Write the compressed frame to the media file. */
    return av_interleaved_write_frame(oFormatContext, pkt);
}

获取经过时间的代码：

qint64 FFCapture::getElapsedTime(qint64 *previousTime) {
    qint64 newTime = timer.elapsed();
    if(newTime > *previousTime) {
        *previousTime = newTime;
        return newTime;
    }
    return -1;
}

添加PTS（分别为视频和音频流）的代码： p>

Code for adding the PTS (video and audio stream, respectively):

qint64 time = getElapsedTime(&previousVideoTime);
if(time >= 0) outFrame->pts = time;
//if(time >= 0) outFrame->pts = av_rescale_q(time, outStream.videoStream->codec->time_base, outStream.videoStream->time_base);

qint64 time = getElapsedTime(&previousAudioTime);
if(time >= 0) {
    AVRational aux;
    aux.num = 1;
    aux.den = 1000;
    outFrame->pts = time;
    //outFrame->pts = av_rescale_q(time, aux, outStream.audioStream->time_base);
}

推荐答案

听起来像你需要给帧（音频和视频）实时时间戳。创建一个函数，返回自从以毫秒（一个整数）开始捕获以来的经过时间。然后将每个流的 time_base 设置为 {1,1000} 并设置 pts 每个框架返回你的函数的值。但要小心：您不能拥有< =以前的时间戳记的时间戳。所以你需要放下框架，如果你几次全部同时处理（或者写另一个机制来处理这种情况）。

Sounds like you need to give the frames (audio and video) real timestamps. Create a function that returns the elapsed time since you started the capture in milliseconds (an integer). Then set time_base for each stream to {1,1000} and set pts of each frame to the return value of your function. But be careful: you can't have a timestamp that is <= a previous timestamp. So you will need to drop frames if you get several all at once (or write another mechanism for dealing with this situation).

从我的更长的答案中摘取这里。

Taken from my longer answer here.

使用 QElapsedTimer 此处。

这篇关于FFMPEG API - 录制视频和音频 - 同步问题的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！