OpenCV，Python：在拼接图像时消除最终的缩小 [英] OpenCV, Python: Eliminating eventual narrowing when stitching images

问题描述

在很大程度上要感谢stackoverflow上的一些很好的答案（

解决方案

< blockquote>

我为什么这样做的理论是相机并不完全垂直于地面。

这是一个很好的直觉。如果摄像机成角度，那么当它朝向物体移动时，该物体在框架中变大。因此，如果您将其拼接到前一帧，则当前帧需要缩小以适合前一帧中的对象。

完整 3x3 单应性包括 x 中的扭曲和 y 指示，但 2x3 仿射转换没有。要坚持使用当前的管道，您可以尝试寻找仿射或欧几里德（刚性）转换。它们之间的区别是仿射经线允许在 x 和 y 指示中分别进行剪切和拉伸，欧几里得仅转换进行平移，旋转和均匀缩放。两者都保留平行线，而完整的单应线则没有，所以你最终会得到一个方形图像变得更加梯形，重复这会缩小你的图像。仿射经线仍然可以在一个方向上缩小，将正方形变成矩形，因此它仍然可能缩小。欧几里德变换只能缩放整个方格，因此仍然可能缩小。

当然，它们不会像 findHomography 或者，但他们应该可以让你关闭匹配而不会扭曲大小。使用OpenCV有两种方法可以找到欧几里德或仿射变换：

1. estimateRigidTransform（） 而不是 warpPerspective（）使用参数 fullAffine = False 进行严格扭曲或使用进行仿射扭曲fullAffine =真。




2. findTransformECC（） ，带有可选参数 motionType = cv2.MOTION_EUCLIDEAN 或 motionType = cv2.MOTION_AFFINE （但是仿射是默认值，所以没有必要指定）。

您可以在文档页面上查看算法之间的差异，或者尝试两者查看最适合您的算法。

如果这也不成功，你可以尝试估算一个扭曲框架的单应性，使其完全垂直于地面。如果您这样做，可以尝试将其应用于所有帧 first ，然后匹配图像。否则，您可能希望转向更高级的方法，而不是在每个帧之间找到单应性。

Thanks in large part to some great answers on stackoverflow (here, here, and here) I've been having some pretty good success in aligning images. There is one issue, though, as you can see below. As I stitch many images together, they get smaller and smaller.

My theory on why this is going on is that the camera wasn't exactly perpendicular to the ground, so as I added more and more images the natural perspective in having a camera not perpendicular to the ground caused the far images to become smaller. This could very well be completely incorrect, though.

However, even when I transform the first image so that it's "as if" it was taken perpendicular to the ground (I think) the distortion still occurs.

Does the brilliant stackoverflow community have any ideas on how I can remedy the situation?

This is the process I use to stitch the images:

1. Using knowledge of the corner lat/long points of images, warp such that the first image is perpendicular to ground. The homography I use to do this is the "base" homography
2. Find common features between each image and the last one using goodFeaturesToTrack() and calcOpticalFlowPyrLK()
3. Use findHomography() to find the homography between the two images. Then, compose that homography with all the previous homographies to to get the "net" homography
4. Apply the transformation and overlay the image with the net result of what I've done so far.

There is one major constraint

The mosaic must be constructed one image at a time, as the camera moves. I am trying to create a real-time map as a drone is flying, fitting each image with the last, one by one.

解决方案

My theory on why this is going on is that the camera wasn't exactly perpendicular to the ground.

This is a good intuition. If the camera is angled, then as it moves towards an object, that object becomes larger in the frame. So if you're stitching that to the previous frame, the current frame needs to shrink to fit to the object in the previous frame.

Full 3x3 homographies include distortions in the x and y directions, but 2x3 affine transformations do not. To stick with your current pipeline, you can try finding an affine or Euclidean (rigid) transformation instead. The difference between them is an affine warp allows for shearing and stretching separately in the x and y directions, Euclidean transforms only do translation, rotation, and uniform scaling. Both preserve parallel lines, whereas a full homography does not, so you could end up with a square image becoming more trapezoidal, and repeating that will shrink your image. An affine warp can still shrink in just one direction, turning a square into a rectangle so it still might shrink. Euclidean transformations can only scale the whole square, so it still might shrink.

Of course, they won't be as perfect matches as findHomography either, but they should be able to get you to close matches without distorting the size as much. There are two options to find Euclidean or affine transformations with OpenCV:

1. estimateRigidTransform() instead of warpPerspective() to get either a rigid warp with the parameter fullAffine=False or an affine warp with fullAffine=True.

2. findTransformECC() with optional parameter motionType=cv2.MOTION_EUCLIDEAN or motionType=cv2.MOTION_AFFINE (but affine is the default so it's not necessary to specify).

You can check out the difference between the algorithms on their documentation pages, or try both to see what works best for you.

If this doesn't work out as well, you can try estimating the homography which warps a frame to be completely perpendicular to the ground. If you do that, you can try applying it to all frames first, and then matching the images. Otherwise, you'll probably want to move to more advanced methods than finding just an homography between each frame.

这篇关于OpenCV，Python：在拼接图像时消除最终的缩小的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！