使用AVPlayer的多个视频 [英] Multiple videos with AVPlayer

问题描述

我正在开发一款适用于iPad的iOS应用，需要在屏幕的某些部分播放视频。我有几个视频文件需要按照编译时未给出的顺序相互播放。它看起来好像只是一个视频播放。从一个视频到下一个视频是很好的，这是两个视频的最后一帧或第一帧显示的延迟，但应该没有闪烁或没有内容的白色屏幕。视频不包含音频。记住内存使用很重要。视频分辨率非常高，可以同时播放几个不同的视频序列。

I am developing an iOS app for iPad that needs to play videos in some part of the screen. I have several video files that needs to be played after each other in an order that is not given at compile time. It must looks as if it is just one video playing. It is fine that when going from one video to the next that is some delay where the last or first frame of the two videos are shown, but there should be no flickering or white screens with no content. The videos does not contain audio. It is important to take memory usage into account. The videos have a very high resolution and several different video sequences can be played next to each other at the same time.

为了获得这个，我尝试了一些解决方案到目前为止。它们列在下面：

In order to obtain this I have tried a few solutions for far. They are listed below:

在这个解决方案中我有一个AVPlayer，它只能在AVPlayerItem上使用包含所有视频的AVComposition制作彼此相邻。当我去特定的视频时，我会在下一个视频开始的构图中寻找时间。这个解决方案的问题是，当寻找玩家时会快速显示它正在寻找的一些帧，这是不可接受的。似乎没有办法直接跳到作曲中的特定时间。我尝试通过制作刚刚完成的视频中最后一帧的图像来解决这个问题，然后在搜索时显示在AVPLayer前面，最后在搜索完成后删除它。我使用AVAssetImageGenerator制作图像，但由于某种原因，图像的质量与视频不同，因此在视频上显示和隐藏图像时会有显着的变化。
另一个问题是AVPlayer使用大量内存，因为单个AVPlayerItem包含所有视频。

In this solution I have an AVPlayer that will use only on AVPlayerItem made using a AVComposition containing all the videos put in next to each other. When going to specific videos I seek to the time in the composition where the next video start.The issue with this solution is that when seeking the player will quickly show some of the frames that it is seeking by, which is not acceptable. It seems that there is no way to jump directly to a specific time in the composition. I tried solving this by making an image of the last frame in the video that just finished, then show that in front of the AVPLayer while seeking, and finally remove it after seeking was done. I am making the image using AVAssetImageGenerator, but for some reason the quality of the image is not the same as the video, so there is notable changes when showing and hiding the image over the video. Another issue is that the AVPlayer uses a lot of memory because a single AVPlayerItem holds all the videos.

此解决方案为每个视频使用AVPlayerItem，并在切换到新视频时替换AVPlayer的项目。这个问题是，当切换AVPlayer的项目时，它会在加载新项目时短时间显示白色屏幕。要解决此问题，可以使用加载时将图像放在最后一帧的前面，但仍然存在图像和视频的质量不同且值得注意的问题。

This solution uses a AVPlayerItem for each video and the replaces the item of the AVPlayer when switching to a new video. The issue with this is that when switching the item of a AVPlayer it will show a white screen for a short time while loading the new item. To fix this the solution with putting an image in front with the last frame while loading could be used, but still with the issue that the quality of image and video is different and notable.

我尝试的下一个解决方案是让两个AVPlayer在彼此之上轮流播放AVPlayerItems。因此，当玩家完成播放时，它将保留在视频的最后一帧。另一个AVPlayer将被带到前面（其项目设置为nil，因此它是透明的），下一个AVPlayerItem将被插入到该AVPlayer中。一旦加载它就会开始播放，两个视频之间顺畅交易的错觉将完好无损。此解决方案的问题是内存使用情况。在某些情况下，我需要同时在屏幕上播放两个视频，这将导致4个AVPlayers同时加载AVPlayerItem。这只是太多的记忆，因为视频的分辨率非常高。

The next solution I tried was having two AVPlayer on top of each other that would take turns playing AVPlayerItems. So When on of the players is done playing it will stay on the last frame of the video. The other AVPlayer will be brought to the front (with its item set to nil, so it is transparent), and the next AVPlayerItem will be inserted in that AVPlayer. As soon as it is loaded it will start playing and the illusion of smooth transaction between the two videos will be intact. The issue with this solution is the memory usage. In some cases I need to play two videos on the screen at the same time, which will result in 4 AVPlayers with a loaded AVPlayerItem at the same time. This is simply too much memory since the videos can be in a very high resolution.

有没有人有一些想法，建议，评论或有关整体问题和上面发布的尝试解决方案。

Does anyone have some thoughts, suggestions, comments or something concerning the overall problem and the tried solutions posted above.

推荐答案

所以该项目现在正在App Store中存在现在是时候回到这个主题并分享我的发现并揭示我最终做的事情。

So the project is now live in the App Store and it is time to come back to this thread and share my findings and reveal what I ended up doing.

什么行不通

我在大型AVComposition上使用其中包含所有视频的第一个选项还不够好，因为无法在没有小的情况下跳转到合成中的特定时间擦洗毛刺。此外，我在组合中的两个视频之间准确暂停视频时遇到问题，因为API无法为暂停提供帧保证。

The first option where I used on big AVComposition with all the videos in it was not good enough since there was no way to jump to a specific time in the composition without having a small scrubbing glitch. Further I had a problem with pausing the video exactly between two videos in the composition since the API could not provide frame guarantee for pausing.

第三个有两个AVPlayers并让他们轮流在实践中工作得很好。特别是在iPad 4或iPhone 5上。具有较低RAM量的设备是一个问题，因为在内存中同时有多个视频消耗了太多内存。特别是因为我不得不处理分辨率非常高的视频。

The third with having two AVPlayers and let them take turns worked great in practice. Exspecially on iPad 4 or iPhone 5. Devices with lower amount of RAM was a problem though since having several videos in memory at the same time consumed too much memory. Especially since I had to deal with videos of very high resolution.

我最终做了什么

好吧，左边是选项号2.在需要时为视频创建AVPlayerItem并将其提供给AVPlayer。这个解决方案的好处是内存消耗。通过延迟创建AVPlayerItem并在不再需要它们的时刻丢弃它们可以将内存消耗降至最低，这对于支持RAM有限的旧设备非常重要。这个解决方案的问题在于，当从一个视频转到另一个视频时，在下一个视频被加载到内存中的同时，会有一个空白屏幕。我解决这个问题的想法是在AVPlayer后面放一个图像，以便在播放器缓冲时显示。我知道我需要的图像与视频完美匹配，因此我捕获的图像是视频的最后一帧和第一帧的精确副本。这个解决方案在实践中运作良好。

Well, left was option number 2. Creating an AVPlayerItem for a video when needed and feeding it to the AVPlayer. The good thing about this solution was the memory consumption. By lazy creating the AVPlayerItems and throwing them away the moment they were not longer needed in could keep memory consumption to a minimum, which was very important in order to support older devices with limited RAM. The problem with this solution was that when going from one video to the next there a blank screen for at quick moment while the next video was loaded into memory. My idea of fixing this was to put an image behind the AVPlayer that would show when the player was buffering. I knew I needed images that we exactly pixel to pixel perfect with the video, so I captured images that were exact copies of the last and first frame of the videos. This solution worked great in practice.

此解决方案的问题

I虽然如果视频/图像不是它的原始大小或模块4的缩放，那么UIImageView内部图像的位置与AVPlayer内部视频的位置不一样。换句话说，我遇到了如何用UIImageView和AVPlayer处理半像素的问题。它似乎没有相同的方式。

I had the issue though that the position of the image inside the UIImageView was not the same as the position of the video inside the AVPlayer if the video/image was not its native size or a module 4 scaling of that. Said other words, I had a problem with how half pixels were handled with a UIImageView and a AVPlayer. It did not seem to be the same way.

我如何解决它

我尝试了很多东西，因为我的应用程序是以交互方式使用视频，以不同的大小显示。我尝试更改AVPlayerLayer和CALayer的magnificationfilter和minificationFilter以使用相同的算法，但没有真正改变任何东西。最后，我最终创建了一个iPad应用程序，可以自动截取我需要的所有尺寸的视频截图，然后在视频缩放到一定大小时使用正确的图像。这使得我在显示特定视频的所有尺寸上都是像素完美的图像。不是一个完美的工具链，但结果是完美的。

I tried a lot of stuff since I my application was using the videos in interactive way where shown in different sizes. I tried changed the magnificationfilter and minificationFilter of AVPlayerLayer and CALayer to use the same algorithm but did not really change anything. In the end I ended up creating an iPad app that automatically could take screenshots of the videos in all the sizes I needed and then use the right image when the video was scaled to a certain size. This gave images that were pixel perfect in all of the sizes I was showing a specific video. Not a perfect toolchain, but the result was perfect.

最终反思

这个位置问题对我来说非常明显（因此非常重要）的主要原因是因为我的应用程序正在播放的视频内容是绘制的动画，其中很多内容处于固定位置且只有部分内容图片正在移动。如果所有内容仅移动一个像素，则会产生非常明显和丑陋的故障。在今年的WWDC上，我与一位Apple工程师讨论了这个问题，他是AVFoundation的专家。当我向他介绍问题时，他的建议基本上是选择3，但我向他解释说这是不可能的，因为内存消耗和我已经尝试过解决方案了。有鉴于此，他说我选择了正确的解决方案，并要求我在缩放视频时为UIImage / AVPlayer定位提交错误报告。

The main reason why this position problem was very visible for me (and therefore very important to solve), was because the video content my app is playing is drawn animations, where a lot the content is in a fixed position and only a part of the picture is moving. If all the content is moved just one pixel it gives a very visible and ugly glitch. At WWDC this year I discussed this problem with an Apple engineer that is an expert in AVFoundation. When I introduced the problem to him his suggestion basically was to go with option 3, but I explained to him that that was not possible because memory consumption and that I already tried that solution out. In that light he said that I choose the right solution and asked me to file a bug report for the UIImage/AVPlayer positioning when video is scaled.

这篇关于使用AVPlayer的多个视频的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！