逐帧合并两个重叠的视频以形成单个帧 [英] Combine two overlapping videos frame by frame to form a single frame

问题描述

我从2个单独的摄像机获取视频输入，输出视频之间存在一定的重叠区域.我已经尝试了将水平输出的视频组合在一起的代码.这是该代码的链接:

I am getting video input from 2 separate cameras with some area of overlap between the output videos. I have tried out a code which combines the video output horizontally. Here is the link for that code:

https://github.com/rajatsaxena/NeuroscienceLab/blob /master/positiontracking/combinevid.py

以可视方式解释该问题:

To explain the problem visually:

红色部分表示两个图像帧之间的重叠区域.我需要输出看起来像第二张图像，第一帧显示为蓝色，第二帧显示为绿色(如第三幅图所示)

The red part shows the overlap region between two image frame. I need the output to look like the second image, with first frame in blue and second frame in green (as shown in third illustration)

我能想到但无法实现的解决方案是，使用SIFT/SURF找出两个帧的最大距离关键点，然后完全拍摄第一个视频帧，然后从第二个视频帧中选取非重叠区域并水平组合他们获得缝合的输出.

A solutions I can think of but unable to implement is, Using SIFT/SURF find out the maximum distance keypoints from both frames and then take the first video frame completely and just pick the non overlapping region from second video frame and horizontally combine them to get the stitched output.

也请让我知道其他解决方案.谢谢！

Let me know of any other solutions possible as well. Thanks!

推荐答案

一个小时前我读了这篇文章.我尝试了一些非常简单的方法.不是很完美，但是在某些情况下应该可以正常工作.例如，如果两个摄像机并排放置在一帧中.

I read this post one hour ago. I tried some really easy approach. Not perfect but in some cases should work well. For example, if you have both cameras on one frame placed side by side.

我从手机上拍摄了2张图像，就像一张照片(彩色图像)一样.程序从源图像中选择矩形"区域并调整大小，然后提取此roi矩形.这个想法是通过归一化相关找到最佳"重叠的Rect区域.

I took 2 images from the phone like on a picture (color images). Program select Rectangles region from both source images and resize end extract this roi rectangles. The idea is to find the "best" overlapping Rect regions by normalized correlation.

M1和M2是可比性， matchTemplate(M1，M2，res，TM_CCOEFF_NORMED);

M1 and M2 is mat roi to compare, matchTemplate(M1, M2, res, TM_CCOEFF_NORMED);

之后，我发现这个重叠的Rect使用它来裁剪源图像并通过hconcat()函数合并在一起.

After, I find this overlapping Rect use this to crop source images and combine by hconcat() function together.

我的代码是C ++，但是在python中复制它确实很简单.它不是最佳解决方案，而是最简单的解决方案之一.如果您的相机之间固定在稳定的位置.我认为这是一个很好的解决方案. 我手握手机:)

My code is in C++ but is really simple to replicate this in python. It is not the best solution but one of the most simple solution. If your cameras are fixed in stable position between themselves. This is a good solution I think. I hold my phone in hand :)

您也可以在视频上使用这种简单的方法.速度仅取决于您比较的矩形候选对象的数量.

You can also use this simple approach on video. The speed depends only on the number of rectangle candidate you compare.

您可以通过智能区域进行改进以比较选择.

You can improve this by smart region to compare selection.

此外，我正在考虑另一种使用光流的方法，即同时将相机中的图像放置在彼此的后方.从一张图像中可能的重叠区域中提取良好的特征，以在第二张图像的区域中进行跟踪和查找.

Also, I am thinking about another idea to use optical flow by putting your images from a camera at the same time to sequence behind each other. From the possible overlapping regions in one image extract good features to track and find them in the region of second images.

对此，冲浪和筛选非常有用，但这是我脑海中最简单的想法.

Surf and sift are great for this but this is the most simple idea on my mind.

代码在这里代码

这篇关于逐帧合并两个重叠的视频以形成单个帧的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！