在R中与fft执行相位相关 [英] Performing a phase correlation with fft in R

问题描述

我正在尝试使用Wikipedia( http://en.wikipedia.org/wiki/Phase_correlation )，以便跟踪2张图片之间的运动.这些图像(帧)是通过相机在风中摇晃而捕获的，最终目的是消除这些帧和后续帧中的晃动.以下是两个示例图像和R代码:

I am trying to implement a 2d phase correlation algorithm in R using a recipe from Wikipedia (http://en.wikipedia.org/wiki/Phase_correlation) in order to track the movement between 2 images. These images (frames) were captured with a camera shaking in the wind and the ultimate goal is to remove the shake in these and subsequent frames. The two example images and the R code are below:

## we will need the tiff library 
library(tiff)

## read in the tiff files 
f1=as.matrix(readTIFF('f1.tiff',native=TRUE))
f2=as.matrix(readTIFF('f2.tiff',native=TRUE))

## take the fft of the first  frame
F1 <- fft(f1)
## take the Conjugate fft of the second frame
F2.c <- Conj(fft(f2))

## calculate the cross power spectrum according to the wiki article
R <- (F1*F2.c)/abs(F1*F2.c)
## take the inverse fft of R
r <- fft(R,inv=TRUE)/length(R)
## because the zero valued imaginary numbers are not needed
r <- Re(r)

## show the normalized cross-correlation
image(r)

## find the max in the cross correlation matrix, or the phase shift -
## between the two images
shift <- which(r==max(r),arr.ind=TRUE)

据我所知，向量 shift 应该包含有关传递移位(dx和dy)的信息，可以最好地校正这两个图像.但是，移位变量给出dx = 1和dy = 1，我认为这表明x或y方向均无移位.对于在x和y方向都有可见位移或几个像素的后续帧，会发生这种情况.

The vector shift, to my understanding, should contain information on the transitive shift (dx and dy) that best corrects these two images. However the shift variable gives dx=1 and dy=1, which I assume indicates no shift in either the x or y direction. This occurs for subsequent frames where there are visible shifts or several pixels in both the x and y direction.

你们所有人中的任何一个都在我的代码/公式中看到错误吗?还是在进行相位相关之前需要先尝试一些像过滤图像之类的东西?

Do any of y'all see an error in my code/formulas? Or do I need to try something fancier like filtering the images first before I do a phase correlation?

加油，伙计们！

推荐答案

根据我对相位相关性的了解，代码看起来是正确的.如果我正确理解了您想要的内容，那么您将尝试使用相位相关来确定两个图像之间的偏移量，因为它们的单应性仅是水平和垂直偏移量.之所以很可能只将偏移视为原点，是因为您的图像缺乏足够的高频信息来正确确定一个良好的偏移.

The code looks correct from what I know about phase correlation. If I understand what you want correctly, you are trying to use phase correlation to determine the offset between two images given that their homographies are nothing more than horizontal and vertical offsets. The fact that you're only getting the shift to be at the origin is most likely due to your images lacking sufficient high frequency information in order to properly determine a good shift.

请改用这两张图片(它们来自您引用的Wikipedia文章，但我将它们提取出来并保存为单独的图片):

Try these two images instead (these were from the Wikipedia article you referenced, but I extracted them out and saved them as individual images):

当我用您的R代码运行这两个图像时，我将其用于相位相关图.请记住，您的图像实际上已另存为.png，因此我不得不将库更改为library(png)，并且我使用了readPNG而不是readTIFF.当尝试使用上面的示例图像运行代码时，请记住这一点:

When I run these two images with your R code, I get this for my phase correlation map. Bear in mind that your images were actually saved as .png, so I had to change the library to library(png) and I used readPNG instead of readTIFF. Keep that in mind when you try and run your code with the above example images:

此外，出现最大峰的位置是:

Also, the location of where the maximum peak occurred was:

> shift
     row col
[1,] 132 153

这告诉我们图像偏移了132行和153列.请注意，这是相对于图像的中心.如果要确定实际偏移，则需要将其减去垂直坐标的一半行和水平坐标的一半列.

This tells us that the image shifted over by 132 rows and 153 columns. Take note that this is with respect to the centre of the image. If you want to determine the actual offset, you'll need to subtract this by half the rows for the vertical coordinate and half the columns for the horizontal coordinate.

因此，该代码完全可以正常工作……只是您的图像缺少足够的高频信息才能使相位相关起作用.在这种情况下，相关性试图做的是，我们试图在每个图像之间找到相似"的变化.如果每个图像之间有很多变化并且非常相似，则相位相关将很好地工作.但是，如果我们没有太大的变化，则相位相关将不起作用.

Therefore, the code works totally fine... it's just that your images lack sufficient high frequency information for the phase correlation to work. What correlation is trying to do in this case is that we're trying to find "similar" variations between each image. If there are a lot of variations between each image and are very similar, then phase correlation will work well. However, if we don't have that much variation, then phase correlation won't work.

为什么会这样?相位相关背后的基础是，我们假设图像被高斯白噪声破坏，因此，如果我们将白噪声与其自身相关(从一个图像到另一个图像)，它将在偏移或偏移处给出一个非常好的高峰值.几乎到处都是零.由于您的图像缺少大量高频信息，并且图像很干净，因此相位相关实际上不起作用.因此，实际上有人建议对图像进行预增白，以使图像包含白噪声，以便在我们要讨论的偏移量处获得漂亮的峰值.

Why is that the case? The basis behind phase correlation is that we assume that the image is corrupted with Gaussian white noise, and so if we correlate white noise with itself (from one image to another) it will give a very nice high peak at where the offset or the shift is and almost zero everywhere. Because of the fact that your images lack a lot of high frequency information and the fact that the images are clean, then phase correlation actually won't work. Therefore, what some people actually suggest is to pre-whiten your image so that the image contains white noise so that you can get the nice peak at where the offset should be that we're talking about.

但是，仅要确保消除任何错误的最大值，最好还对频域中的互相关矩阵(R代码中的r)进行平滑处理，以使只有一个真正的最大值.在频域/FFT域中使用高斯滤波器应该可以正常工作.

However, just to make sure that you eliminate any false maximums, it is a good idea to also smoothen the cross-correlation matrix in the frequency domain (r in your R code) so that there is a high probability that there will only be one true maximum. Using a Gaussian filter in the frequency / FFT domain should work fine.

无论如何，我看不出您的图片有多少变化，因此可以避免的是，您必须确保您的图片具有很多高频信息才能起作用！

In any case, I don't see much variation in your images and so something to take away from this is that you gotta make sure your image has a lot of high frequency information for this to work!

这篇关于在R中与fft执行相位相关的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！