如何使用多个AVVideoCompositions合成视频 [英] How to composite videos using multiple AVVideoCompositions

问题描述

我正在尝试找出如何将多个视频(AVAssets)合成为单个视频，以使每个视频都经历自己的视频组成.但是，我看不到有什么方法可以实现，并且想知道是否有人有任何想法.

I'm trying to figure out how to composite multiple videos (AVAssets) into a single video such that each of the videos goes through its own video composition. However, I can't see a way to accomplish this and was wondering if anyone had any ideas.

请考虑以下内容:

上面的图片说明了我正在尝试做的事情.我想从四个不同的视频中获取视频轨道，并将它们合并为一个视频，以便它们以网格状布局播放.

The above picture illustrates what I'm trying to do. I want to take the video track from four different videos and merge them into a single video such that they play in a grid-like layout.

现在，我可以通过执行以下操作来实现此目标

Right now, I'm able to achieve this by doing the following

• 创建一个AVMutableComposition

将四个视频轨道添加到可变构图

Add four video tracks to the mutable composition

创建四个AVMutableVideoCompositionLayerInstruction实例，并应用相应的transform来缩放和平移轨迹.

Create four AVMutableVideoCompositionLayerInstruction instances with the appropriate transform applied to scale and translate the track.

创建一个AVMutableVideoComposition，向其添加图层说明，然后在相应的AVAssetExportSession

Create an AVMutableVideoComposition, add the layer instructions to it and then set that video composition on the appropriate AVAssetExportSession

尽管这可行，但是在将视频合成到输出剪辑之前，它无法为我提供对每个视频轨道进行任何自定义效果的能力.例如，在上图中，我想在每个轨道周围添加唯一的边框颜色.对于单个视频(因此没有分层说明)，我可以使用AVVideoComposition applyingCIFiltersWithHandler之类的东西添加边框，效果很好.但是我不能使用它，因为我需要视频合成来设置转换和缩放指令.

While this works, it doesn't offer me the ability to do any custom effects on each video track before compositing it into the output clip. For example, in the picture above I'd like to add a unique border color around each track. With a single video (and thus no layer instructions), then I can just add the border using something like AVVideoComposition applyingCIFiltersWithHandler, which works great. But I can't use that because I need the video composition for setting the transform and scale instructions.

(applyingCIFiltersWithHandler似乎没有提供一种方法来访问构成要渲染的帧的轨道.)

(applyingCIFiltersWithHandler doesn't appear to offer a way to get access to the tracks that make up the frame being rendered.)

我想我想要的是能够在AVMutableVideoCompositionLayerInstruction中执行某种类型的自定义渲染的功能，但是当前唯一可用的选项是transform和opacity.

What I think I want is the ability to do some type of custom rendering within a AVMutableVideoCompositionLayerInstruction, but the only options currently available are transform and opacity ones.

(请注意，任何解决方案都要求我在渲染每个视频轨道时都能够访问当前的合成时间 ，因为我想渲染的某些效果"是帧计数器和时间码显示.)

(Note that any solution requires that I have access to the current composition time during rendering for each video track because some of the "effects" I'd like to render are a frame counter and a timecode display.)

任何帮助，想法或技巧都将不胜感激.

Any help, ideas or tips would be appreciated.

推荐答案

您需要实现实现AVVideoCompositing协议的自定义合成器. https://developer.apple.com/reference/avfoundation/avvideocompositing

You need to implement a custom compositor that implements the AVVideoCompositing protocol. https://developer.apple.com/reference/avfoundation/avvideocompositing

该协议中的startRequest函数接收 https://developer.apple.com/reference/avfoundation/avasynchronousvideocompositionrequest ，这使您可以从每个轨道访问信息 https ://developer.apple.com/reference/avfoundation/avasynchronousvideocompositionrequest/1390379-sourceframe ，然后将它们与您自己的自定义指令组合在一起，并以您自己的方式进行合成(将它们组合在一起)，返回具有框架以您喜欢的方式与轨道合成.

The startRequest function within that protocol receives an https://developer.apple.com/reference/avfoundation/avasynchronousvideocompositionrequest and this allows you to access the information from each track https://developer.apple.com/reference/avfoundation/avasynchronousvideocompositionrequest/1390379-sourceframe, then combine them with your own custom instructions and composite (put them together) in your own way, returning a final single buffer that has the frame composited from the tracks in the way you like.

这篇关于如何使用多个AVVideoCompositions合成视频的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！