如何检测图像之间的偏移 [英] How to detect a shift between images

问题描述

我正在分析多个图像，需要能够判断它们与参考图像相比是否发生了偏移.目的是判断相机是否在拍摄图像之间移动.理想情况下，我希望能够纠正偏移以便仍然进行分析，但至少我需要能够确定图像是否偏移并在超过某个阈值时将其丢弃.

I am analyzing multiple images and need to be able to tell if they are shifted compared to a reference image. The purpose is to tell if the camera moved at all in between capturing images. I would ideally like to be able to correct the shift in order to still do the analysis, but at a minimum I need to be able to determine if an image is shifted and discard it if it's beyond a certain threshold.

以下是我想要检测的图像偏移的一些示例:

Here are some examples of the shifts in an image I would like to detect:

我将使用第一张图像作为参考，然后将以下所有图像与它进行比较，以确定它们是否发生了偏移.图像是灰度的(它们只是使用热图以彩色显示)并存储在二维 numpy 数组中.任何想法我怎么能做到这一点?我更喜欢使用我已经安装的包(scipy、numpy、PIL、matplotlib).

I will use the first image as a reference and then compare all of the following images to it to figure out if they are shifted. The images are gray-scale (they are just displayed in color using a heat-map) and are stored in a 2-D numpy array. Any ideas how I can do this? I would prefer to use the packages I already have installed (scipy, numpy, PIL, matplotlib).

推荐答案

正如 Lukas Graf 提示的那样，您正在寻找互相关.它运行良好，如果:

As Lukas Graf hints, you are looking for cross-correlation. It works well, if:

1. 图像的比例不会发生很大变化.
2. 图像没有旋转变化.
3. 图像中没有显着的光照变化.

对于简单的翻译，互相关非常好.

For plain translations cross-correlation is very good.

最简单的互相关工具是scipy.signal.correlate.但是，它使用了互相关的平凡方法，对于边长为 n 的二维图像，该方法为 O(n^4).实际上，使用您的图像需要很长时间.

The simplest cross-correlation tool is scipy.signal.correlate. However, it uses the trivial method for cross-correlation, which is O(n^4) for a two-dimensional image with side length n. In practice, with your images it'll take very long.

更好的是 scipy.signal.fftconvolve 因为卷积和相关性密切相关.

The better too is scipy.signal.fftconvolve as convolution and correlation are closely related.

像这样:

import numpy as np
import scipy.signal

def cross_image(im1, im2):
   # get rid of the color channels by performing a grayscale transform
   # the type cast into 'float' is to avoid overflows
   im1_gray = np.sum(im1.astype('float'), axis=2)
   im2_gray = np.sum(im2.astype('float'), axis=2)

   # get rid of the averages, otherwise the results are not good
   im1_gray -= np.mean(im1_gray)
   im2_gray -= np.mean(im2_gray)

   # calculate the correlation image; note the flipping of onw of the images
   return scipy.signal.fftconvolve(im1_gray, im2_gray[::-1,::-1], mode='same')

im2_gray[::-1,::-1] 有趣的索引将其旋转 180°(水平和垂直镜像).这就是卷积和相关的区别，相关是与镜像的第二个信号的卷积.

The funny-looking indexing of im2_gray[::-1,::-1] rotates it by 180° (mirrors both horizontally and vertically). This is the difference between convolution and correlation, correlation is a convolution with the second signal mirrored.

现在，如果我们只是将第一个(最上面的)图像与其自身相关联，我们得到:

Now if we just correlate the first (topmost) image with itself, we get:

这给出了图像自相似性的度量.最亮的点位于 (201, 200)，它位于 (402, 400) 图像的中心.

This gives a measure of self-similarity of the image. The brightest spot is at (201, 200), which is in the center for the (402, 400) image.

可以找到最亮的点坐标:

The brightest spot coordinates can be found:

np.unravel_index(np.argmax(corr_img), corr_img.shape)

最亮像素的线性位置由argmax返回，但必须使用unravel_index将其转换回二维坐标.

The linear position of the brightest pixel is returned by argmax, but it has to be converted back into the 2D coordinates with unravel_index.

接下来，我们通过将第一张图像与第二张图像相关联来尝试相同的操作:

Next, we try the same by correlating the first image with the second image:

相关性图像看起来相似，但最佳相关性已移动到 (149,200)，即图像中向上 52 个像素.这是两个图像之间的偏移量.

The correlation image looks similar, but the best correlation has moved to (149,200), i.e. 52 pixels upwards in the image. This is the offset between the two images.

这似乎适用于这些简单的图像.但是，也可能存在错误的相关峰值，并且本答案开头概述的任何问题都可能会破坏结果.

This seems to work with these simple images. However, there may be false correlation peaks, as well, and any of the problems outlined in the beginning of this answer may ruin the results.

在任何情况下，您都应该考虑使用窗口函数.函数的选择没那么重要，只要用到东西就行.此外，如果您遇到小旋转或缩放变化的问题，请尝试将几个小区域与周围图像相关联.这将在图像的不同位置为您提供不同的位移.

In any case you should consider using a windowing function. The choice of the function is not that important, as long as something is used. Also, if you have problems with small rotation or scale changes, try correlating several small areas agains the surrounding image. That will give you different displacements at different positions of the image.

这篇关于如何检测图像之间的偏移的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！