使用AutoGen FFmpeg库在MP4中同步音频/视频 [英] Sync Audio/Video in MP4 using AutoGen FFmpeg library

问题描述

我正在使用的AVCodecContexts：

对于视频：

  AVCodec * videoCodec = ffmpeg.avcodec_find_encoder（AVCodecID.AV_CODEC_ID_H264）
 AVCodecContext * videoCodecContext = ffmpeg.avcodec_alloc_context3（videoCodec）; 
 videoCodecContext-> bit_rate = 400000; 
 videoCodecContext-> width = 1280; 
 videoCodecContext-> height = 720; 
 videoCodecContext-> gop_size = 12; 
 videoCodecContext-> max_b_frames = 1; 
 videoCodecContext-> pix_fmt = videoCodec-> pix_fmts [0]; 
 videoCodecContext-> codec_id = videoCodec-> id; 
 videoCodecContext-> codec_type = videoCodec-> type; 
 videoCodecContext-> time_base = new AVRational 
 {
 num = 1，
 den = 30 
}; 
  

对于音频：

  AVCodec * audioCodec = ffmpeg.avcodec_find_encoder（AVCodecID.AV_CODEC_ID_AAC）
 AVCodecContext * audioCodecContext = ffmpeg.avcodec_alloc_context3（audioCodec）; 
 audioCodecContext-> bit_rate = 1280000; 
 audioCodecContext-> sample_rate = 48000; 
 audioCodecContext-> channels = 2; 
 audioCodecContext-> channel_layout = ffmpeg.AV_CH_LAYOUT_STEREO; 
 audioCodecContext-> frame_size = 1024; 
 audioCodecContext-> sample_fmt = audioCodec-> sample_fmts [0]; 
 audioCodecContext-> profile = ffmpeg.FF_PROFILE_AAC_LOW; 
 audioCodecContext-> codec_id = audioCodec-> id; 
 audioCodecContext-> codec_type = audioCodec-> type; 
  

在编写视频帧时，我设置PTS的位置如下：

  outputFrame-> pts = frameIndex; //当前图像帧的索引是
  

然后我使用avcodec_encode_video2（）对帧进行编码。之后，我打电话给以下设置时间戳：

  ffmpeg.av_packet_rescale_ts（& packet，videoCodecContext-> time_base ，videoStream-> time_base）; 
  

这完美地播放。

当我做同样的音频时，视频播放缓慢的动作，播放音频，然后随身携带视频，没有声音。

我找不到例如在MP4文件中如何设置视频/音频的pts / dts位置的任何地方。任何帮助的例子将是非常好的！

另外，我正在写视频帧，之后（一旦他们都写），我写的音频。我已经在评论中建议的调整值更新了这个问题。

我已经上传了一个测试视频来显示我的结果： http://www.filedropper.com/test_124

解决方案

解决问题。我已经添加了一个新功能来设置PTS位置之后设置视频/音频位置。

视频只是通常的增量（每帧为+1），而音频完成如下：

  outputFrame-> pts = ffmpeg.av_rescale_q（m_audioFrameSampleIncrement，new AVRational {num = 1，den = 48000}，m_audioCodecContext-> time_base）; 
 
 m_audioFrameSampleIncrement + = outputFrame-> nb_samples; 
  

框架编码后，我调用我的新功能：

  private static void SetPacketProperties（ref AVPacket packet，AVCodecContext * codecContext，AVStream * stream）
 {
 packet.pts = ffmpeg.av_rescale_q_rnd .pts，codecContext-> time_base，stream-> time_base，AVRounding.AV_ROUND_NEAR_INF | AVRounding.AV_ROUND_PASS_MINMAX）; 
 packet.dts = ffmpeg.av_rescale_q_rnd（packet.dts，codecContext-> time_base，stream-> time_base，AVRounding.AV_ROUND_NEAR_INF | AVRounding.AV_ROUND_PASS_MINMAX）; 
 packet.duration =（int）ffmpeg.av_rescale_q（packet.duration，codecContext-> time_base，stream-> time_base）; 
 packet.stream_index = stream-> index; 
} 
  

I'm currently having problems making my audio and video streams stay synced.

These are the AVCodecContexts I'm using:

For Video:

AVCodec* videoCodec = ffmpeg.avcodec_find_encoder(AVCodecID.AV_CODEC_ID_H264)
AVCodecContext* videoCodecContext = ffmpeg.avcodec_alloc_context3(videoCodec);
videoCodecContext->bit_rate = 400000;
videoCodecContext->width = 1280;
videoCodecContext->height = 720;
videoCodecContext->gop_size = 12;
videoCodecContext->max_b_frames = 1;
videoCodecContext->pix_fmt = videoCodec->pix_fmts[0];
videoCodecContext->codec_id = videoCodec->id;
videoCodecContext->codec_type = videoCodec->type;
videoCodecContext->time_base = new AVRational
{
    num = 1,
    den = 30
};

For Audio:

AVCodec* audioCodec = ffmpeg.avcodec_find_encoder(AVCodecID.AV_CODEC_ID_AAC)
AVCodecContext* audioCodecContext = ffmpeg.avcodec_alloc_context3(audioCodec);
audioCodecContext->bit_rate = 1280000;
audioCodecContext->sample_rate = 48000;
audioCodecContext->channels = 2;
audioCodecContext->channel_layout = ffmpeg.AV_CH_LAYOUT_STEREO;
audioCodecContext->frame_size = 1024;
audioCodecContext->sample_fmt = audioCodec->sample_fmts[0];
audioCodecContext->profile = ffmpeg.FF_PROFILE_AAC_LOW;
audioCodecContext->codec_id = audioCodec->id;
audioCodecContext->codec_type = audioCodec->type;

When writing the video frames, I setup the PTS position as follows:

outputFrame->pts = frameIndex;  // The current index of the image frame being written

I then encode the frame using avcodec_encode_video2(). After this, I call the following to setup the time stamps:

ffmpeg.av_packet_rescale_ts(&packet, videoCodecContext->time_base, videoStream->time_base);

This plays perfectly.

However, when I do the same for audio, the video plays in slow motion, plays the audio first and then carry's on with the video afterwards with no sound.

I cannot find an example anywhere of how to set pts/dts positions for video/audio in an MP4 file. Any examples of help would be great!

Also, I'm writing the video frames first, after which (once they are all written) I write the audio. I've updated this question with the adjusted values suggested in the comments.

I've uploaded a test video to show my results here: http://www.filedropper.com/test_124

解决方案

Solved the problem. I've added a new function to set video/audio positions after setting the frames PTS positions.

Video is just the usual increment (+1 for each frame), whereas audio is done as follows:

outputFrame->pts = ffmpeg.av_rescale_q(m_audioFrameSampleIncrement, new AVRational { num = 1, den = 48000 }, m_audioCodecContext->time_base);

m_audioFrameSampleIncrement += outputFrame->nb_samples;

After the frame is encoded, I call my new function:

private static void SetPacketProperties(ref AVPacket packet, AVCodecContext* codecContext, AVStream* stream)
{
    packet.pts = ffmpeg.av_rescale_q_rnd(packet.pts, codecContext->time_base, stream->time_base, AVRounding.AV_ROUND_NEAR_INF | AVRounding.AV_ROUND_PASS_MINMAX);
    packet.dts = ffmpeg.av_rescale_q_rnd(packet.dts, codecContext->time_base, stream->time_base, AVRounding.AV_ROUND_NEAR_INF | AVRounding.AV_ROUND_PASS_MINMAX);
    packet.duration = (int)ffmpeg.av_rescale_q(packet.duration, codecContext->time_base, stream->time_base);
    packet.stream_index = stream->index;
}

这篇关于使用AutoGen FFmpeg库在MP4中同步音频/视频的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！