FFmpeg-视频的pts和dts不能正确增加，音频的ptmp和dts不能正确增加 [英] FFmpeg - pts and dts not increasing properly for video, but does for audio
问题描述
我正在尝试拍摄两个视频，然后将它们组合成一个视频.但是，当我运行代码时，在解码/编码第二个视频时出现此错误:
I am trying to take two videos and put them together into one video. However, when I run my code, I get this error when decoding/encoding the second video:
Application provided invalid, non monotonically increasing dts to muxer in stream 0
代码完成后，第一个视频完全没问题，但是第二个视频却没问题.我能够产生的最佳结果是第二个视频的后半部分紧接在第一个视频之后.有趣的是，音频完全可以正常播放.
When the code finishes, the first video is completely fine, but the second video is not. The best result that I have been able to produce is where the second half of the second video comes right after the first video. The funny thing is, the audio is completely fine and is as it should be.
过去，我已经能够使用解码代码来简单地复制视频(视频和音频).
In the past, I have been able to use my decoding/encoding code to simply copy a video (both video and audio).
我已经在网上搜索了有关此特定问题的信息，并尝试了建议的解决方案，但似乎都没有一个能解决我的问题.这些是我一直在查看的线程:
I have searched online regarding this specific problem and tried the suggested solutions, but none of them seem to fix my issue. These are the threads that I have been looking at:
如何使用libavformat用相同的编解码器合并2个视频文件(重新混合)?
这是我编写的当前代码:
This is the current code that I have written:
视频和ClipSequence结构:
Video and ClipSequence structs:
typedef struct Video {
char* filename;
AVFormatContext* inputContext;
AVFormatContext* outputContext;
AVCodec* videoCodec;
AVCodec* audioCodec;
AVStream* inputStream;
AVStream* outputStream;
AVCodecContext* videoCodecContext_I; // Input
AVCodecContext* audioCodecContext_I; // Input
AVCodecContext* videoCodecContext_O; // Output
AVCodecContext* audioCodecContext_O; // Output
int videoStream;
int audioStream;
SwrContext* swrContext;
} Video;
typedef struct ClipSequence {
VideoList* videos;
AVFormatContext* outputContext;
AVStream* outputStream;
int64_t lastpts, lastdts;
int64_t currentpts, currentdts;
} ClipSequence;
解码和编码(相同的音频代码):
Decoding and encoding (same code for audio):
int decodeVideoSequence(ClipSequence* sequence, Video* video, AVPacket* packet, AVFrame* frame) {
int response = avcodec_send_packet(video->videoCodecContext_I, packet);
if (response < 0) {
printf("[ERROR] Failed to send video packet to decoder\n");
return response;
}
while (response >= 0) {
response = avcodec_receive_frame(video->videoCodecContext_I, frame);
if (response == AVERROR(EAGAIN) || response == AVERROR_EOF) {
break;
} else if (response < 0) {
printf("[ERROR] Failed to receive video frame from decoder\n");
return response;
}
if (response >= 0) {
// Do stuff and encode
sequence->currentpts = packet->pts; // Store decoded packet's pts and dts
sequence->currentdts = packet->dts;
if (encodeVideoSequence(sequence, video, frame) < 0) {
printf("[ERROR] Failed to encode new video\n");
return -1;
}
}
av_frame_unref(frame);
}
return 0;
}
int encodeVideoSequence(ClipSequence* sequence, Video* video, AVFrame* frame) {
AVPacket* packet = av_packet_alloc();
if (!packet) {
printf("[ERROR] Could not allocate memory for video output packet\n");
return -1;
}
int response = avcodec_send_frame(video->videoCodecContext_O, frame);
if (response < 0) {
printf("[ERROR] Failed to send video frame for encoding\n");
return response;
}
while (response >= 0) {
response = avcodec_receive_packet(video->videoCodecContext_O, packet);
if (response == AVERROR(EAGAIN) || response == AVERROR_EOF) {
break;
} else if (response < 0) {
printf("[ERROR] Failed to receive video packet from encoder\n");
return response;
}
// Set packet to have pts and dts based on the previous video's pts and dts
packet->flags |= AV_PKT_FLAG_KEY;
packet->pts = sequence->currentpts + sequence->lastpts;
packet->dts = sequence->currentdts + sequence->lastdts;
packet->stream_index = video->videoStream;
packet->duration = 1000; // 60 fps
response = av_interleaved_write_frame(sequence->outputContext, packet);
if (response < 0) {
printf("[ERROR] Failed to write video packet\n");
break;
}
}
av_packet_unref(packet);
av_packet_free(&packet);
return 0;
}
阅读框架:
int readSequenceFrames(ClipSequence* sequence, Video* video, AVPacket* packet, AVFrame* frame) {
if (!packet) {
printf("[ERROR] Packet not allocated to be read\n");
return -1;
}
if (!frame) {
printf("[ERROR] Frame not allocated to be read\n");
return -1;
}
// Sets video and audio codec context parameters
if (prepareVideoOutStream(video) < 0) {
printf("[ERROR] Failed to prepare output video stream\n");
return -1;
}
if (prepareAudioOutStream(video) < 0) {
printf("[ERROR] Failed to prepare output audio stream\n");
return -1;
}
// Prepares audio resampling
if (initResampler(video->audioCodecContext_I, video->audioCodecContext_O, &(video->swrContext)) < 0) {
printf("[ERROR] Failed to init audio resampler\n");
return -1;
}
// Read packets
int frameNum = 0;
while (av_read_frame(video->inputContext, packet) >= 0) {
printf("[READ] Reading frame %i\n", frameNum);
if (packet->stream_index == video->videoStream) {
if (decodeVideoSequence(sequence, video, packet, frame) < 0) {
printf("[ERROR] Failed to decode and encode video\n");
return -1;
}
} else if (packet->stream_index == video->audioStream) {
if (decodeAudioSequence(sequence, video, packet, frame) < 0) {
printf("[ERROR] Failed to decode and encode audio\n");
return -1;
}
}
av_packet_unref(packet);
frameNum++;
}
// Increment pts and dts by the last pts and dts in the current video
sequence->lastpts += sequence->currentpts;
sequence->lastdts += sequence->currentdts;
return 0;
}
我认为增加pts和dts时我有正确的逻辑.我不确定我到底想念什么.
I believe that I have the right logic when I am increasing the pts and dts. I am not sure what exactly that I am missing.
谢谢.
推荐答案
// Increment pts and dts by the last pts and dts in the current video
sequence->lastpts += sequence->currentpts;
sequence->lastdts += sequence->currentdts;
这是错误的.首先，暂时忽略PTS，仅处理DTS.
This is wrong. First, ignore PTS for now and only deal with DTS.
DTS不是相对数字，而是绝对数字.将递增数字加在一起会创建一个指数序列.例如:如果这是一个30 fps的视频，并且时基是1/30，则DTS的每一帧都将前进一帧.例如0、1、2、3、4、5、6、7、9
DTS is not a relative number, it's an absolute number. Adding incrementing numbers together creates an exponential sequence. For example: if this is a 30 fps video, and the time base is 1/30, the every frame the DTS will advance by one. e.g. 0, 1, 2, 3, 4, 5, 6, 7, 9
如果继续将它们添加在一起，您将获得:0、1、3、6、10、15、21、28、36、45
If you keep adding them together, you will get: 0, 1, 3, 6, 10, 15, 21, 28, 36, 45
因此， sequence-> lastdts =序列-> currentdts; 不是 sequence->> lastdts + =序列-> currentdts;
设置新的DTS时，您需要添加帧时长例如 packet-> dts = sequence-> lastdts + frame_duration;
When you set the new DTS, you need to add in a frame duration
e.g. packet->dts = sequence->lastdts + frame_duration;
否则，如果需要多一帧，则该帧将具有与前一帧相同的DTS
Otherwise this frame will have the same DTS as the previous frame, when it needs to be 1 frame more
下一个PTS:
PTS不是单调的，并且会在时间上倒退.您也无法以这种方式跟踪它，因为下一个PTS的时间戳可能更短.为了解决这个问题，您需要将PTS转换为CTS，然后返回:
PTS is not monotonic and can go backwards in time. You and can not track it this way because the next PTS may be a lower time stamp. To solve this you need to convert PTS to CTS, and then back:
auto cts = packet->pts - packet->dts
packet->dts = // Whatever you are updating the dts to
packet->pts = packet->dts + cts
还在每帧上设置 packet->标志| = AV_PKT_FLAG_KEY; 会在搜索时引起问题
Also setting packet->flags |= AV_PKT_FLAG_KEY; on every frame will cause problems when seeking
这篇关于FFmpeg-视频的pts和dts不能正确增加，音频的ptmp和dts不能正确增加的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
