将AVAssetWriter与原始NAL单元一起使用 [英] Using AVAssetWriter with raw NAL Units

问题描述

我在iOS文档中注意到 AVAssetWriterInput 您可以为 outputSettings传递 nil 字典，指定不应重新编码输入数据。

I noticed in the iOS documentation for AVAssetWriterInput you can pass nil for the outputSettings dictionary to specify that the input data should not be re-encoded.

用于编码媒体的设置附加到输出。传递nil以指定不应重新编码附加的样本。

The settings used for encoding the media appended to the output. Pass nil to specify that appended samples should not be re-encoded.

我想利用此功能传入流原始H.264 NAL，但我无法将我的原始字节流调整到 CMSampleBuffer ，我可以将其传递到AVAssetWriterInput的 appendSampleBuffer 方法。我的NAL流仅包含SPS / PPS / IDR / P NAL（1,5,7,8）。我无法找到有关如何使用AVAssetWriter预编码H264数据的文档或结论性答案。生成的视频文件无法播放。

I want to take advantage of this feature to pass in a stream of raw H.264 NALs, but I am having trouble adapting my raw byte streams into a CMSampleBuffer that I can pass into AVAssetWriterInput's appendSampleBuffer method. My stream of NALs contains only SPS/PPS/IDR/P NALs (1, 5, 7, 8). I haven't been able to find documentation or a conclusive answer on how to use pre-encoded H264 data with AVAssetWriter. The resulting video file is not able to be played.

如何将NAL单元正确打包到 CMSampleBuffers ？我是否需要使用起始码前缀？长度前缀？我是否需要确保每个 CMSampleBuffer 只放一个NAL？我的最终目标是使用H264 / AAC创建一个MP4或MOV容器。

How can I properly package the NAL units into CMSampleBuffers? Do I need to use a start code prefix? A length prefix? Do I need to ensure I only put one NAL per CMSampleBuffer? My end goal is to create an MP4 or MOV container with H264/AAC.

这是我一直在玩的代码：

Here's the code I've been playing with:

-(void)addH264NAL:(NSData *)nal
{
    dispatch_async(recordingQueue, ^{
        //Adapting the raw NAL into a CMSampleBuffer
        CMSampleBufferRef sampleBuffer = NULL;
        CMBlockBufferRef blockBuffer = NULL;
        CMFormatDescriptionRef formatDescription = NULL;
        CMItemCount numberOfSampleTimeEntries = 1;
        CMItemCount numberOfSamples = 1;


        CMVideoFormatDescriptionCreate(kCFAllocatorDefault, kCMVideoCodecType_H264, 480, 360, nil, &formatDescription);
        OSStatus result = CMBlockBufferCreateWithMemoryBlock(kCFAllocatorDefault, NULL, [nal length], kCFAllocatorDefault, NULL, 0, [nal length], kCMBlockBufferAssureMemoryNowFlag, &blockBuffer);
        if(result != noErr)
        {
            NSLog(@"Error creating CMBlockBuffer");
            return;
        }
        result = CMBlockBufferReplaceDataBytes([nal bytes], blockBuffer, 0, [nal length]);
        if(result != noErr)
        {
            NSLog(@"Error filling CMBlockBuffer");
            return;
        }
        const size_t sampleSizes = [nal length];
        CMSampleTimingInfo timing = { 0 };
        result = CMSampleBufferCreate(kCFAllocatorDefault, blockBuffer, YES, NULL, NULL, formatDescription, numberOfSamples, numberOfSampleTimeEntries, &timing, 1, &sampleSizes, &sampleBuffer);

        if(result != noErr)
        {
            NSLog(@"Error creating CMSampleBuffer");
        }
        [self writeSampleBuffer:sampleBuffer ofType:AVMediaTypeVideo];
    });
}

请注意，我正在调用 CMSampleBufferSetOutputPresentationTimeStamp 在 writeSampleBuffer 方法内的示例缓冲区中，我认为这是我实际尝试追加它之前的有效时间。

Note that I'm calling CMSampleBufferSetOutputPresentationTimeStamp on the sample buffer inside of the writeSampleBuffer method with what I think is a valid time before I'm actually trying to append it.

感谢任何帮助。

推荐答案

我设法让视频播放工作在VLC但不是QuickTime的。我使用类似于上面发布的代码将H.264 NAL转换为CMSampleBuffers。

I managed to get video playback working in VLC but not QuickTime. I used code similar to what I posted above to get H.264 NALs into CMSampleBuffers.

我有两个主要问题：

1. 我没有正确设置CMSampleTimingInfo（正如我上面的评论所述）。


2. 我没有正确打包原始NAL数据（不确定这是哪里记录，如果在任何地方）。

要解决＃1，我设置 timing.duration = CMTimeMake（1 ，fps）; 其中fps是预期的帧速率。然后我设置 timing.decodeTimeStamp = kCMTimeInvalid; 表示将按解码顺序给出样本。最后，我通过计算绝对时间设置 timing.presentationTimeStamp ，我也用 startSessionAtSourceTime 。

To solve #1, I set timing.duration = CMTimeMake(1, fps); where fps is the expected frame rate. I then set timing.decodeTimeStamp = kCMTimeInvalid; to mean that the samples will be given in decoding order. Lastly, I set timing.presentationTimeStamp by calculating the absolute time, which I also used with startSessionAtSourceTime.

要解决＃2，通过反复试验，我发现以下列形式提供的NAL单位有效：

To solve #2, through trial and error I found that giving my NAL units in the following form worked:

[7 8 5] [1] [1] [1]..... [7 8 5] [1] [1] [1]..... (repeating)

其中每个NAL单元的前缀是32位起始码，等于 0x00000001 。

Where each NAL unit is prefixed by a 32-bit start code equaling 0x00000001.

大概是因为它没有在QuickTime中播放，我仍然无法将生成的.mov文件移动到相册（ ALAssetLibrary 方法 videoAtPathIsCompatibleWithSavedPhotosAlbum 未能说明电影无法播放。希望有人知道会发生什么可以评论。谢谢！

Presumably for the same reason it's not playing in QuickTime, I'm still having trouble moving the resulting .mov file to the photo album (the ALAssetLibrary method videoAtPathIsCompatibleWithSavedPhotosAlbum is failing stating that the "Movie could not be played." Hopefully someone with an idea about what's going on can comment. Thanks!

这篇关于将AVAssetWriter与原始NAL单元一起使用的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！