将视频分割为多个场景（片段）的最佳方法是什么？ [英] What is the best way to divide a video into scenes (segments)
问题描述
我已被要求拍摄一个给定的视频，可能是一个简单的卡通，并返回一系列场景。
我需要使用opencv库做到这一点，结果格式是无关的（即我可以返回每个场景的时间或实际分割视频）。
I've been requested to take a given video, probably a simple cartoon, and return an array of its scenes.
I need to use the opencv libary in order to do it, and the result format is irrelevent (i.e. I can return the timespans of each scene or actualy split the video).
任何帮助都会被认同。
感谢
推荐答案
从技术上讲， 场景< a> 是在单个位置连续合在一起的组 。 镜头是视频的基本叙事元素，由多个
Technically, a scene is a group shots which are successively taken together at a single location. A shot is a basic narrative element of the video which is composed of a number of frames that are presented from a continuous viewpoint.
将视频自动分割为其镜头被称为镜头边界检测问题，其中基本思想是识别形成从一个镜头到另一镜头的转换的连续帧。
Automatically dividing a video into its shots is called the shot boundary detection problem in which the basic idea is identifying consecutive frames that form a transition from one shot to another.
标识转换通常涉及计算两个帧之间的相似性值。可以使用低级图像特征（例如颜色，边缘或运动）来计算该值。简单的相似性指标可以是：
Identifying transitions generally involve calculating a similarity value between two frames. This value can be calculated using low level image features such as color, edge or motion. A simple similarity metric could be:
- s（ f 1 ， f i 在所有像素位置中都是
/ strong> 颜色（ i ） - f strong>））/ N
- s(f1, f2) = sum(i in all pixel locations)(abs(ficolor(i) - f2color(i))) / N
f2 表示两个不同的视频帧， N 表示这些帧中的数字像素。这是两个帧之间的平均第一阶（曼哈顿）像素颜色距离。
where f1 and f2 represent two distinct video frames and N represents number pixels in those frames. This is the average first order (Manhattan) pixel color distance between two frames.
假设您有一个由框架组成的视频 { f 1 ，f 2 f } ，并计算了相邻帧之间的距离。简单的决定措施可以标记从 f a 到 f b 作为镜头边界的转变如果（ f a ， f b ）低于某个。
Say you have a video composed of frames { f1, f2 ... fM } and you have calculated this distance between neighboring frames. A simple decision measure could be labeling a transition from fa to fb as a shot boundary if s(fa, fb) is below a certain threshold.
成功的镜头边界检测器使用二阶（或更多）距离，例如距离或欧氏距离
A successful shot boundary detector uses distances of second order (or more) such as Euclidean distance or Pearson correlation coefficient and utilizes a combination of different features instead of using only one, say color.
通常，相机或物体移动会破坏帧之间的像素对应关系。在直方图的帮助下使用低级细节的频率将在这里解决。
Usually, a camera or object movement breaks the pixel correspondence between frames. Using frequencies of low level details with the help of histograms will be a cure here.
此外，在两个以上的框架上执行决策有助于找到平滑转换 替换一段时间。确定一组帧还有助于我们识别由闪光灯或快速移动摄像头引起的假转换。
Also, performing decision making over more than two frames helps in finding smooth transitions where one shot dissolves into or replaces another for a duration. Deciding for a group of frames also help us in identifying false transitions caused by light flashes or fast moving cameras.
如果您遇到问题，请开始从基本方法，如比较RGB颜色和视频帧之间的边缘响应。一起分析您的结果和数据，尝试调整新的功能，距离指标和决策方法，以获得更好的效果。
For your problem, please start from basic approaches like comparing RGB colors and edge responses between video frames. Analyze your results and data together and try to adapt new features, distance metrics and decision making methods for better performance.
将视频细分为多个镜头的最佳方法取决于您的数据。机器学习方法，例如用高斯混合模型进行概率建模框架转换或分类通过 支持向量机 预计效果优于手动选择的阈值。但是，您必须在有效选择输入功能之前学习基础知识。
The best way of segmenting a video into shots will vary depending on your data. Machine learning approaches like probabilistically modeling frame transitions with Gaussian mixture models or classification through support vector machines are expected to perform better than hand selected thresholds. However it is important that you learn the basics before effectively choosing input features.
自动查找镜头边界将足以将您的视频分成有意义的部分。另一方面，将您的视频划分为场景被认为是更难的语义问题。然而，镜头分割是它的第一步。
Automatically finding shot boundaries will be sufficent to divide your video into meaningful parts. Dividing your video into scenes, on the other hand, is considered a harder semantic problem. Nevertheless, shot segmentation is the first step to it.
这篇关于将视频分割为多个场景（片段）的最佳方法是什么？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
