使用AVAssetReader从远程资产中读取（流） [英] Using AVAssetReader to read (stream) from a remote asset

问题描述

我的主要目标是从服务器流式传输视频，并在流式传输时逐帧剪切（以便OpenGL可以使用它）。为此，我使用了我在互联网上随处可见的代码（我记得它来自Apple的GLVideoFrame示例代码）：

My main goal is to stream a video from a server, and cut it frame by frame while streaming (so that it can be used by OpenGL). For that, I've used this code that I found everywhere on the Internet (as I recall it was from Apple's GLVideoFrame sample code):

NSArray * tracks = [asset tracks];
NSLog(@"%d", tracks.count);

for(AVAssetTrack* track in tracks) {

    NSLog(@"type: %@", [track mediaType]);

    initialFPS = track.nominalFrameRate;
    width = (GLuint)track.naturalSize.width;
    height = (GLuint)track.naturalSize.height;


    NSError * error = nil;

    // _movieReader is a member variable
    @try {
        self._movieReader = [[[AVAssetReader alloc] initWithAsset:asset error:&error] autorelease];
    }
    @catch (NSException *exception) {
        NSLog(@"%@ -- %@", [exception name], [exception reason]);
        NSLog(@"skipping track");

        continue;
    }


    if (error)
    {
        NSLog(@"CODE:%d\nDOMAIN:%@\nDESCRIPTION:%@\nFAILURE_REASON:%@", [error code], [error domain], error.localizedDescription, [error localizedFailureReason]);                                          
        continue;
    }

    NSString* key = (NSString*)kCVPixelBufferPixelFormatTypeKey;
    NSNumber* value = [NSNumber numberWithUnsignedInt:kCVPixelFormatType_32BGRA];
    NSDictionary* videoSettings = [NSDictionary dictionaryWithObject:value forKey:key]; 
    [_movieReader addOutput:[AVAssetReaderTrackOutput assetReaderTrackOutputWithTrack:track
                                                                       outputSettings:videoSettings]];
    [_movieReader startReading];
    [self performSelectorOnMainThread:@selector(frameStarter) withObject:nil waitUntilDone:NO];
}

但我总是在 [[AVAssetReader]处得到此异常alloc] initWithAsset：error：] 。

NSInvalidArgumentException -- *** -[AVAssetReader initWithAsset:error:] Cannot initialize an instance of AVAssetReader with an asset at non-local URL 'http://devimages.apple.com/iphone/samples/bipbop/bipbopall.m3u8'

所以我的两个问题是：

1. 这个例外是否真的有用我 AVAssetReader 必须有本地网址？它可以用于流媒体（就像 AVFoundation 类的其余部分一样）？


2. 如果 AVFoundation 方法不起作用，有什么其他建议可以同时流式传输视频并拆分帧？

1. Is the exception really telling me that AVAssetReader must have a local URL? Can it be used for streaming (just like the rest of the AVFoundation classes)?
2. If the AVFoundation approach won't work, what are other suggestions to stream the video and split its frames at the same time?

非常感谢你的帮助。

推荐答案

@Cocoanetics，由于低代表我无法发表评论。但你所说的似乎并不完全正确。 AVFoundation似乎没有区分本地和非本地文件，因为它在使用的KIND文件或协议之间。使用mp4 / mov与通过m3u8使用HTTP Live流协议之间有一个非常明显的区别，但使用本地或远程mp4的差异有点模糊。

@Cocoanetics, I'm unable to comment due to low rep. But what you've said does not seem to be entirely correct. AVFoundation does not seem to distinguish as much between local and non-local files, as it does between the KIND of files or protocols used. There is a VERY clear distinction between using mp4/mov's versus using the HTTP Live streaming protocol via m3u8's, but the differences using a local or remote mp4 are a little fuzzier.

要扩展上述内容：

a）如果您的远程资产是M3U8（即，你正在使用HTTP'实时'流媒体），然后没有任何机会。无论M3U8是在本地文件系统中还是在远程服务器上，由于多种原因，AVAssetReader和所有与AVAsset相关的功能都无法正常工作。 但是，AVPlayer，AVPlayerItem等可以正常工作。

a) If your 'remote' asset is an M3U8 (that is, you are using HTTP 'live' streaming), then no chance whatsoever. No matter if the M3U8 is in your local filesystem or on a remote server, for a multitude of reasons AVAssetReader and all AVAsset-associated functionality just does NOT work. However, AVPlayer, AVPlayerItem etc would work just fine.

b）如果是MP4 / MOV，几乎没有进一步的调查。 本地MP4 / MOV的工作完美无缺。在远程MP4 / MOV的情况下，我能够创建（或从AVPlayerItem或AVPlayer或AVAssetTracks中检索）AVURLAsset我有时能够成功初始化AVAssetReader（我很快就会扩展'有时'）。但是，在远程MP4的的情况下，copyNextSampleBuffer总是返回nil。由于UPTO调用copyNextSampleBuffer的工作有几点，所以我不是100％肯定：

b) If it is an MP4/MOV, a little further investigation is due. Local MP4/MOV's work flawlessly. While in case of remote MP4/MOV's, I'm able to create (or retrieve from an AVPlayerItem or AVPlayer or AVAssetTracks) an AVURLAsset with which I'm sometimes able to initialize an AVAssetReader successfully (I'll expand on the 'sometimes' as well, shortly). HOWEVER, copyNextSampleBuffer always returns nil in case of remote MP4's. Since several things UPTO the point of invoking copyNextSampleBuffer work, I'm not 100% sure if:

i ）copyNextSampleBuffer不能用于远程mp4，在所有其他步骤成功之后，是预期/预期的功能。

i) copyNextSampleBuffer not working for remote mp4's, after all the other steps having been successful, is intended/expected functionality.

ii）其他步骤似乎适用于所有用于远程MP4的都是Apple实施的意外，当我们点击copyNextSampleBuffer ..............这些其他步骤是什么时，这种不兼容性就会突然出现，我会详细说明很快。

ii) That the 'other steps' seem to work at all for remote MP4's is an accident of Apple's implementation, and this incompatibility is simply coming to the fore when we hit copyNextSampleBuffer..............what these 'other steps' are, I'll detail shortly.

iii）尝试为远程MP4调用copyNextSampleBuffer时，我做错了。

iii) I'm doing something wrong when trying to invoke copyNextSampleBuffer for remote MP4's.

所以@Paula您可以尝试使用远程MOV / MP4进一步调查。

So @Paula you could try to investigate a little further with remote MOV/MP4's.

供参考，以下是我尝试从视频中捕获帧的方法：

For reference, here are the approaches I tried for capturing a frame from videos:

a）

a)

直接从视频网址创建AVURLAsset。

Create an AVURLAsset directly from the video URL.

使用[asset tracksWithMediaType：AVMediaTypeVideo]检索视频轨道

Retrieve the video track using [asset tracksWithMediaType:AVMediaTypeVideo]

使用视频轨道准备AVAssetReaderTrackOutput资源。

Prepare an AVAssetReaderTrackOutput using the video track as the source.

使用AVURLAsset创建AVAssetReader。

Create an AVAssetReader using the AVURLAsset.

将AVAssetReaderTrackOutput添加到AVAssetReader和startReading。

Add AVAssetReaderTrackOutput to the AVAssetReader and startReading.

使用copyNextSampleBuffer检索图像。

Retrieve images using copyNextSampleBuffer.

b）

从视频URL创建AVPlayerItem，然后从中创建AVPlayer（或直接从URL创建AVPlayer）。

Create an AVPlayerItem from the video URL, and then an AVPlayer from it (or create the AVPlayer directly from the URL).

检索AVPlayer的'asset'属性并使用loadValuesAsynchronouslyForKeys：加载其'track'。

Retrieve the AVPlayer's 'asset' property and load its 'tracks' using "loadValuesAsynchronouslyForKeys:".

将AVMediaTypeVideo类型的曲目分开（或者在加载曲目后简单地在资源上调用tracksWithMediaType：），然后使用视频曲目创建AVAssetReaderTrackOutput。

Separate the tracks of type AVMediaTypeVideo (or simply call tracksWithMediaType: on the asset once the tracks are loaded), and create your AVAssetReaderTrackOutput using the video track.

使用AVPlayer的'asset'，'startReading'创建AVAssetReader，然后使用copyNextSampleBuffer检索图像。

Create AVAssetReader using the AVPlayer's 'asset', 'startReading' and then retrieve images using copyNextSampleBuffer.

c）

直接从视频网址创建AVPlayerItem + AVPlayer或AVPlayer。

Create an AVPlayerItem+AVPlayer or AVPlayer directly from the video URL.

KVO AVPlayerItem的'tracks'属性，加载曲目后，将AVMediaTypeVideo类型的AVAssetTracks分开。

KVO the AVPlayerItem's 'tracks' property, and once the tracks are loaded, separate the AVAssetTracks of type AVMediaTypeVideo.

从AVPlayerItem / AVPlayer / AVAssetTrack的'asset'属性中检索AVAsset。

Retrieve the AVAsset from AVPlayerItem/AVPlayer/AVAssetTrack's 'asset' property.

其余步骤与方法（b）类似。

Remaining steps are similar to approach (b).

d）

直接从视频网址创建AVPlayerItem + AVPlayer或AVPlayer。

Create an AVPlayerItem+AVPlayer or AVPlayer directly from the video URL.

KVO AVPlayerItem的'tracks'属性，加载曲目后，将AVMediaTypeVideo类型分开。

KVO the AVPlayerItem's 'tracks' property, and once the tracks are loaded, separate the ones of type AVMediaTypeVideo.

创建AVMutableComposition，并初始化AVMediaTypeVideo类型的关联AVMutableCompositionTrack。

Create an AVMutableComposition, and initialize an associated AVMutableCompositionTrack of type AVMediaTypeVideo.

将之前检索到的视频轨道中的相应CMTimeRange插入此AVMutableCompositionTrack。

Insert the appropriate CMTimeRange from video track retrieved earlier, into this AVMutableCompositionTrack.

与（b）和（c）类似，现在创建AVAssetReader和AVAssetReaderTrackOutput，但不同之处在于您使用AVMutableComposition作为初始化AVAssetReader的基本AVAsset，和AVMutableCompositionTrack作为AVAssetReaderTrackOutput的基本AVAssetTrack。

Similar to (b) and (c), now create your AVAssetReader and AVAssetReaderTrackOutput, but with the difference that you use the AVMutableComposition as the base AVAsset for initializing your AVAssetReader, and AVMutableCompositionTrack as the base AVAssetTrack for your AVAssetReaderTrackOutput.

'startReading'并使用copyNextSampleBuffer从AVAssetReader获取帧。

'startReading' and use copyNextSampleBuffer to get frames from the AVAssetReader.

PS：我尝试过（d）这里是为了解决直接从AVPlayerItem或AVPlayer检索到的AVAsset没有表现的事实。所以我想从我手头的AVAssetTracks创建一个新的AVAsset。不可否认，也许毫无意义（如果不是最初的AVAsset，最终还是会从哪里获取赛道信息！）但是无论如何都值得绝望。

P.S: I tried approach (d) here to get around the fact that the AVAsset retrieved directly from AVPlayerItem or AVPlayer was not behaving. So I wanted to create a new AVAsset from the AVAssetTracks I already had in hand. Admittedly hacky, and perhaps pointless (where else would the track information be ultimately retrieved from if not the original AVAsset!) but it was worth a desperate try anyway.

以下是不同类型文件的结果摘要：

Here's a summary of the results for different types of files:

1）本地MOV / MP4 - 所有4种方法都能正常运行。

1) Local MOV/MP4's - All 4 approaches work flawlessly.

2）远程MOV / MP4 - 在方法（b）到（d）中正确检索资产和轨道，并且AVAssetReader也被初始化，但copyNextSampleBuffer总是返回nil。在（a）的情况下，AVAssetReader本身的创建失败，出现'未知错误'NSOSStatusErrorDomain -12407。

2) Remote MOV/MP4's - The asset and tracks are retrieved correctly in approaches (b) through (d), and the AVAssetReader is initialized as well but copyNextSampleBuffer always returns nil. In case of (a), creation of the AVAssetReader itself fails with an 'Unknown Error' NSOSStatusErrorDomain -12407.

3）本地M3U8（通过应用程序内访问/本地HTTP服务器） - 方法（a），（b）和（c）失败，因为尝试以任何形状或形式获取通过M3U8流传输的文件的AVURLAsset / AVAsset是一个愚蠢的错误。

3) Local M3U8's (accessed through an in-app/local HTTP server) - Approaches (a), (b) and (c) fail miserably as trying to get an AVURLAsset/AVAsset in any shape or form for files streamed via M3U8's is a fools errand.

如果是（a），则根本不创建资产，并且AVURLAsset上的initWithURL：调用失败并显示未知错误 'AVFoundationErrorDomain -11800。

In case of (a), the asset is not created at all, and the initWithURL: call on AVURLAsset fails with an 'Unknown Error' AVFoundationErrorDomain -11800.

在（b）和（c）的情况下，从AVPlayer / AVPlayerItem或AVAssetTracks中检索AVURLAsset会返回SOME对象，但始终访问它上面的'tracks'属性返回一个空数组。

In case of (b) and (c), retrieving the AVURLAsset from the AVPlayer/AVPlayerItem or AVAssetTracks returns SOME object, but accessing the 'tracks' property on it always returns an empty array.

在（d）的情况下，我能够成功检索和隔离视频轨道，但在尝试创建AVMutableCompositionTrack时，它在尝试插入时失败了CMTimeRange从源轨道进入AVMutableCompositionTrack，带有'未知错误'NSOSStatusErrorDomain -12780。

In case of (d), I'm able to retrieve and isolate the video tracks successfully, but while trying to create the AVMutableCompositionTrack, it fails when trying to insert the CMTimeRange from the source track into the AVMutableCompositionTrack, with an 'Unknown Error' NSOSStatusErrorDomain -12780.

4）远程M3U8，表现完全正确和当地的M3U8相同。

4) Remote M3U8's, behave exactly the same as local M3U8's.

我没有完全了解为什么存在这些差异，或者Apple无法减轻这些差异。但是你去了。

I'm not entirely educated on why these differences exist, or could not have been mitigated by Apple. But there you go.

这篇关于使用AVAssetReader从远程资产中读取（流）的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！