如何使用 FFT 根据局部方向和密度调整具有不同 Gabor 滤波器的图像? [英] How to convolve an image with different gabor filters adjusted according to the local orientation and density using FFT?
问题描述
我目前正在开发一个使用 SFinGe 方法(由 Maltoni、Maio 和 Cappelli 提供)链接生成合成指纹的库:reve-etrange 解释了如何将不同的 gabor 过滤器应用于图像，但问题是他的做法是将不同的过滤器应用于整个图像，然后对响应求和，以及我所做的需要的是不同像素对不同过滤器的响应.
这是我将一个过滤器与图像卷积时发生的情况(使用 FFT):
这是使用的过滤器:
这是它卷积后的图像:
这是FFT实现的C#算法:
//////使用 FFT 对图像进行卷积.///</总结>///<param name="image">要过滤的图片.</param>///<param name=方向图">方向图.</param>///<param name="densityMap">密度图.</param>///如果设置为<c>true</c>[FFT].///<returns></returns>公共双[，]过滤器(双[，]图像，双[，]方向图，双[，]密度图，布尔FFT){double[,] filter = null;double[,] paddedFilter = null;double[,] paddedImage = null;double[,]croppedImage = null;double[,]filteredImage = new double[image.GetLength(0), image.GetLength(1)];double[,]filteredImageWithValuesScaled = new double[image.GetLength(0), image.GetLength(1)];double[,] finalImage = new double[image.GetLength(0), image.GetLength(1)];filter = this.filterBank[70][0];填充过滤器 = PadImage(过滤器, 512, 512, 0, 0);//填充过滤器以具有二维的效力.填充图像 = PadImage(图像, 512, 512, 0, 0);//填充图像以具有二维的效力.FFT fftOne = 新的 FFT(填充图像)；FFT fftTwo = 新的 FFT(填充过滤器)；fftOne.ForwardFFT();fftTwo.ForwardFFT();FFT 结果 = fftOne * fftTwo;结果.InverseFFT();过滤图像 = 结果.GreyImage;FilteredImageWithValuesScaled = this.RescaleValues(filteredImage, 0.0, 255.0);croppedImage = CropImage(filteredImageWithValuesScaled, image.GetLength(0), image.GetLength(1));返回裁剪图像；} 所以我要问的是，您如何使用 FFT 获得从不同像素到不同内核的响应?如果这是不可能的，有没有办法改进我的直接卷积，使其至少快 20 倍?
是否也可以使用所有过滤器制作一个内核，以便我可以将它们应用于整个图像?
我找到了一种方法，可以使用不同的 gabor 滤波器对图像进行卷积，并使用 FFT 根据像素的局部特征收集像素的响应.
这称为上下文过滤，通常当您过滤图像时，您只会将单个内核应用于整个事物，但在上下文过滤中，过滤器的特征会根据局部上下文而变化，在这种情况下，密度和方向像素.
在直接卷积中，过程非常简单，您只需在卷积的每一步更改内核，但在 FFT 卷积中，由于内核应用于频域中的图像，您无法在此期间更改滤波器属性过程.因此，您这样做的方法是分别将图像与每个过滤器进行卷积，这将提供 N 个过滤图像，其中 N 是过滤器组中过滤器的数量，然后您必须构建最终图像，从中获取信息基于您正在重新创建的像素的上下文的不同过滤图像.
因此，对于每个像素，您查看其方向和密度属性，然后从过滤图像中获取该像素位置的值，该图像是通过将原始图像与具有相同属性的过滤器卷积生成的.以下是该过程的示例:
这是方向图的图形表示.
我对所有像素使用相同的密度以减少生成的过滤器数量.
这是源图片:
这是使用的三个过滤器的示例(0 度、45 度、90 度):
以下是源图像与不同过滤器进行不同程度卷积的三个示例:
最后这是生成的图像，图像是根据像素的方向和密度从不同过滤图像中获取像素值创建的.
这个过程比直接卷积慢很多 =(，因为你必须先用所有过滤器对原始图像进行卷积.生成最终图像大约需要一分钟.到目前为止，我似乎坚持使用直接卷积=/.
感谢阅读.
I'm currently working on a library to generate synthetic fingerprints using the SFinGe method (by Maltoni, Maio and Cappelli) link :http://biolab.csr.unibo.it/research.asp?organize=Activities&select=&selObj=12&pathSubj=111%7C%7C12&
One of the steps requires me to apply different gabor filters to an image, each pixel in the image have an orientation and a frequency associated, so the convolution is not done with one kernel over the entire image but the filter must change during the process depending on those attributes of the pixels, that way each pixel on the image is altered in a different way.
If you apply the filters this way, and convolve the image several times(you also have to binarize the image after each convolution) you obtain this:
A master fingerprint, this image took about 20 seconds to be generated (which is way too slow, this is why I want to do it with FFT), since I had to perform 5 times the convolution to complete it (you start from a few random black points).
My filters are 30x30, and the image is 275x400. There are a total of 36000 filters, one for each degree and density (density goes from 0 to 100). I'm planing on reducing the number of filters from 36000 to 9000 since I can cover all the angles with those. Also all the filters are pre-calculated and stored in a filter bank.
This is the source code in C# of the gabor convolution implementation:
This two methods execute the convolution:
/// <summary>
/// Convolve the image with the different filters depending on the orientation and density of the pixel.
/// </summary>
/// <param name="image">The image to be filtered.</param>
/// <param name="directionalMap">The directional map.</param>
/// <param name="densityMap">The density map.</param>
/// <returns></returns>
public double[,] Filter(double[,] image, double[,] directionalMap, double[,] densityMap)
{
int midX = FILTER_SIZE / 2;
int midY = FILTER_SIZE / 2;
double[,] filteredImage = new double[image.GetLength(0), image.GetLength(1)];
double[,] filteredImageWithValuesScaled = new double[image.GetLength(0), image.GetLength(1)];
double[,] finalImage = new double[image.GetLength(0), image.GetLength(1)];
for (int i = 0; i < image.GetLength(0); i++)
for (int j = 0; j < image.GetLength(1); j++)
{
double pixelValue = GetPixelConvolutionValue(image, this.filterBank[(int)Math.Floor((directionalMap[i, j] * 180 / Math.PI))][Math.Round(densityMap[i, j], 2)], i - midX, j - midY);
filteredImage[i, j] = pixelValue;
}
filteredImageWithValuesScaled = this.RescaleValues(filteredImage, 0.0, 255.0);
return filteredImageWithValuesScaled;
}
/// <summary>
/// Gets the pixel convolution value.
/// </summary>
/// <param name="image">The image.</param>
/// <param name="filter">The filter.</param>
/// <param name="sourceX">The source X.</param>
/// <param name="sourceY">The source Y.</param>
/// <returns></returns>
private double GetPixelConvolutionValue(double[,] image, double[,] filter, int sourceX, int sourceY)
{
double result = 0.0;
int totalPixels = 0;
for (int i = 0; i < filter.GetLength(0); i++)
{
if(i + sourceX < 0 || i + sourceX >= image.GetLength(0))
continue;
for (int j = 0; j < filter.GetLength(1); j++)
{
if(j + sourceY < 0 || j + sourceY >= image.GetLength(1))
continue;
double deltaResult = image[sourceX + i,sourceY + j] * filter[i, j];
result += deltaResult;
++totalPixels;
}
}
double filteredValue = result / totalPixels;
return filteredValue;
}
This two methods generate the different gabor filters for the filter bank:
/// <summary>
/// Creates the gabor filter.
/// </summary>
/// <param name="size">The size.</param>
/// <param name="angle">The angle.</param>
/// <param name="wavelength">The wavelength.</param>
/// <param name="sigma">The sigma.</param>
/// <returns></returns>
public double[,] CreateGaborFilter(int size, double angle, double wavelength, double sigma)
{
double[,] filter = new double[size, size];
double frequency = 7 + (100 - (wavelength * 100)) * 0.03;
int windowSize = FILTER_SIZE/2;
for (int y = 0; y < size; ++y)
{
for (int x = 0; x < size; ++x)
{
int dy = -windowSize + y;
int dx = -windowSize + x;
filter[x, y] = GaborFilterValue(dy, dx, frequency, angle, 0, sigma, 0.80);
}
}
return filter;
}
/// <summary>
/// Gabor filter values generation.
/// </summary>
/// <param name="x">The x.</param>
/// <param name="y">The y.</param>
/// <param name="lambda">The wavelength.</param>
/// <param name="theta">The orientation.</param>
/// <param name="phi">The phaseoffset.</param>
/// <param name="sigma">The gaussvar.</param>
/// <param name="gamma">The aspectratio.</param>
/// <returns></returns>
double GaborFilterValue(int x, int y, double lambda, double theta, double phi, double sigma, double gamma)
{
double xx = x * Math.Cos(theta) + y * Math.Sin(theta);
double yy = -x * Math.Sin(theta) + y * Math.Cos(theta);
double envelopeVal = Math.Exp(-((xx * xx + gamma * gamma * yy * yy) / (2.0f * sigma * sigma)));
double carrierVal = Math.Cos(2.0f * (float)Math.PI * xx / lambda + phi);
double g = envelopeVal * carrierVal;
return g;
}
My goal is to reduce this time to under 1 second (There are several programs that do the exactly same thing in such time). So since the direct convolution approach is not working for me I decide to implement the Fast Fourier Transform Convolution, but the problem with this is that FFT applies the same kernel to the entire image at once, and I need to change the kernel per pixel, because each pixel must be altered depending on his attributes (density and orientation). In this post How to apply Gabor wavelets to an image? reve-etrange explains how to apply different gabor filters to an image, but the thing is that the way he does it, is applying the different filters to the whole image and then sum the responses , and what I need is the responses from different pixels to the different filters.
This is whats happening when I convolve one filter with the image (using FFT):
This was the filter used:
And this was the image it was convolved with:
This is the algorithm in C# of the FFT implementation:
/// <summary>
/// Convolve the image using FFT.
/// </summary>
/// <param name="image">The image to be filtered.</param>
/// <param name="directionalMap">The directional map.</param>
/// <param name="densityMap">The density map.</param>
/// <param name="FFT">if set to <c>true</c> [FFT].</param>
/// <returns></returns>
public double[,] Filter(double[,] image, double[,] directionalMap, double[,] densityMap, bool FFT)
{
double[,] filter = null;
double[,] paddedFilter = null;
double[,] paddedImage = null;
double[,] croppedImage = null;
double[,] filteredImage = new double[image.GetLength(0), image.GetLength(1)];
double[,] filteredImageWithValuesScaled = new double[image.GetLength(0), image.GetLength(1)];
double[,] finalImage = new double[image.GetLength(0), image.GetLength(1)];
filter = this.filterBank[70][0];
paddedFilter = PadImage(filter, 512, 512, 0, 0); // Pad the filter to have a potency of 2 dimensions.
paddedImage = PadImage(image, 512, 512, 0, 0); // Pad the image to have a potency of 2 dimensions.
FFT fftOne = new FFT(paddedImage);
FFT fftTwo = new FFT(paddedFilter);
fftOne.ForwardFFT();
fftTwo.ForwardFFT();
FFT result = fftOne * fftTwo;
result.InverseFFT();
filteredImage = result.GreyImage;
filteredImageWithValuesScaled = this.RescaleValues(filteredImage, 0.0, 255.0);
croppedImage = CropImage(filteredImageWithValuesScaled, image.GetLength(0), image.GetLength(1));
return croppedImage;
}
So what I'm asking is, how do you get the response from different pixels to different kernels with FFT? If this is not possible, is there a way to improve my direct convolution to make it at least 20 times faster?
Also would it be possible to make one kernel using all the filters, so I can apply those to the whole image?
I found a way to convolve the image with different gabor filters and gather the responses of the pixels base on their local characteristics using FFT.
This is call contextual filtering, usually when you filter an image you only apply a single kernel to the entire thing, but in contextual filtering the characteristics of the filter change according to the local context, in this case, density and orientation of the pixel.
In direct convolution the process is pretty straight forward, you simply change the kernel at each step of the convolution, but in FFT convolution since the kernel is applied to the image in the frequency domain you can't change the filter properties during the process. So the way you do it is by making the convolution of the image with each filter separately, this will give N number of filtered images where N is the numbers of filters in your filter bank, then you have to construct your final image taking information from the different filtered images based on the context of the pixel you are recreating.
So for each pixel, you look at his orientation and density properties and then grab the value of that pixel position from the filtered image that was generated by the convolving the original image with the filter with those same properties. Here is an example of the process:
This is a graphic representation of the directional map.
I used the same density for all pixels to reduce the amount of filters generated.
This is the source image:
This is an example of three of the filters used (0 degrees, 45 degrees, 90 degrees):
These are three examples of the source image being convolve with the different filters at different degrees:
And finally this is the resulting image, the image was created taking the values of the pixels from the different filtered images base on the direction and density of the pixel.
This process is A LOT slower than the direct convolution =(, since you have to convolve the original image with all the filters first. The final image took about a minute to be generated. So far I'm stuck with the direct convolution it seems =/.
Thanks for reading.
这篇关于如何使用 FFT 根据局部方向和密度调整具有不同 Gabor 滤波器的图像?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
