图像去模糊 [英] Deblurring an image

问题描述

我正在尝试对Python中的图像进行模糊处理，但是遇到了一些问题。这是我尝试过的方法，但请记住，我不是该主题的专家。根据我的理解，如果您知道点扩散函数，则应该能够通过执行反卷积相当简单地对图像进行模糊处理。但是，这似乎不起作用，我也不知道自己是在做愚蠢的事情还是只是对事情的理解不正确。在马克·纽曼（Mark Newman）的《计算物理》一书（使用Python）中，他谈到了问题7.9。在这个问题中，他提供了使用高斯点扩展函数（psf）故意模糊的图像，并且该问题的目的是使用高斯对图像进行模糊处理。这是通过将模糊图像的2D FFT除以psf的2D FFT，然后进行逆变换来实现的。

为了解决这个问题，我想对使用故意失焦的相机拍摄的真实图像进行模糊处理。因此，我安装了相机并拍摄了两组照片。第一组照片聚焦。第一个是在完全昏暗的房间里的很小的LED灯，第二个是上面有文字的纸（使用闪光灯）。然后，在不更改任何距离或任何内容的情况下，我更改了相机的焦点设置，以使文本非常不清晰。然后，我使用闪光灯拍摄了文字图片，并为LED拍摄了第二张图片（没有闪光灯）。这是模糊的图像。

现在，根据我的理解，模糊点光源的图像应该是点扩散函数，因此我应该能够用它来模糊我的形象。问题是，当我这样做时，我得到的图像看起来像是噪点。经过一些研究，使用去卷积技术似乎似乎是一个大问题。但是，考虑到我已经测量出我认为是确切的点扩散函数，我感到惊讶的是噪声在这里是个问题。

我尝试过的一件事是用1或epsilon替换psf变换中的小值（小于epsilon），我尝试了大范围的epsilon值。这样产生的图像不仅是噪点，而且不是图像的去模糊版本。它看起来像是原始（非模糊）图像的怪异，模糊的版本。这是我程序中的图像（您可以忽略sigma的值，该程序中未使用它的值）。

我相信我正在处理噪声问题，但是我不知道为什么，也不知道该怎么办。任何建议将不胜感激（请记住，我不是该领域的专家）。

请注意，我故意不发布代码，因为我认为这有点在这一点上无关紧要。但是，如果有人认为这样做会有用，我将很乐意这样做。我不认为这是一个编程问题，因为我使用了相同的技术，并且当我拥有已知的点扩展函数时（例如，当我将原始对焦图像的FFT除以外焦-聚焦图像，然后进行逆变换）。我只是不明白为什么我似乎无法使用实验测量的点扩散函数。

解决方案

您遇到的问题不幸的是，要解决的问题比您预期的要困难得多。让我分四个部分进行说明。第一部分假定您对傅立叶变换感到满意。

1. 为什么不能通过简单的反卷积来解决此问题。


2. 概述如何执行图像去模糊。


3. 通过FFT进行反卷积以及为什么这是个坏主意


4. 一个反卷积的另一种方法

但首先要注意一些符号：

我用 I 表示图像，用 K 表示卷积核。 I * K 是图像 I 与内核 K 的卷积。 F （I）是图像 I 和 F 的（n维）傅立叶变换（K）是卷积核 K 的傅立叶变换（也称为点扩展函数或PSF）。同样， Fi 是傅立叶逆变换。

为什么无法通过简单的反卷积解决此问题：

当您说我们可以用 Ib 的傅立叶变换除以 K 的傅立叶变换，恢复模糊图像 Ib = I * K 。但是，镜头模糊不是卷积模糊操作。这是一种改进的卷积模糊操作，其中模糊内核 K 取决于与您拍摄的物体的距离。因此，内核会从一个像素变为另一个像素。

您可能会认为这与图像无关，因为您已经在像素位置测量了正确的内核图片。但是，情况可能并非如此，因为图像的较远部分会影响图像的较近部分。解决此问题的一种方法是裁剪图像，使其仅在可见的纸张上。

为什么通过FFT进行反卷积不是一个好主意：

卷积定理指出 I * K = Fi （ F （I） F （K））。该定理导致一个合理的假设，即如果我们有一个图像 Ib = I * K 被卷积核 K 模糊，那么我们可以通过以下方法恢复去模糊的图像：计算 I =（ F （Ib）/ F （K））。

在我们了解为什么这是一个坏主意之前，我想对卷积定理的含义有一些直觉。当我们将图像与内核卷积时，这与获取图像的频率分量并将其与内核的频率分量相乘是一样的。

现在，让我解释一下为什么很难用FFT对图像进行卷积处理。默认情况下，模糊会删除高频信息。因此， K 的高频必须接近零。原因是 I 的高频信息在模糊时会丢失-因此， Ib 的高频分量必须接近零。为此， K 的高频分量也必须趋近于零。

由于 K 的高频分量几乎为零，我们看到 Ib的高频分量当我们对FFT解卷积时，被显着放大（因为我们几乎被零除）。在无噪声的情况下这不是问题。

但是，在嘈杂的情况下，这是一个问题。其原因是，按照定义，噪声就是高频信息。因此，当我们尝试对 Ib 进行反卷积时，噪声几乎被无限放大。这就是使用FFT解卷积的想法不好的原因。

此外，您需要考虑基于FFT的卷积算法如何处理边界条件。通常，当我们对图像进行卷积时，分辨率会有所降低。这是不必要的行为，因此我们引入了边界条件，这些条件指定了图像外部像素的像素值。此类边界条件的示例为

1. 图像外部的像素与图像内部最近的像素具有相同的值


2. 图像外部的像素具有恒定值（例如0）


3. 该图像是周期信号的一部分，因此最上面一行的像素行等于最下面一行的像素

最终边界条件通常对一维信号有意义。但是，对于图像来说，这毫无意义。不幸的是，卷积定理指定使用周期性边界条件。

执行反卷积的另一种方法

由于所有图像都嘈杂，现在已经失去了希望，去卷积会增加噪点。但是，事实并非如此，因为我们有迭代方法来执行去卷积。让我开始向您展示最简单的迭代方法。

让 || I ||²是所有 I 像素的平方和。解方程

Ib = I * K

关于 I 等效于解决以下优化问题：

min L（I）= min || I * K-Ib || ²

关于 I 。这可以使用梯度下降来完成，因为 L 的梯度由下式给出：

DL = Q *（I * K- （em）

其中， Q 是通过转置 K 获得的内核（也称为信号处理文献中的匹配滤波器）。

因此，您可以获得以下将对图像进行模糊处理的迭代算法。

  from scipy.ndimage导入卷积
 
 blurd_image =＃加载图像
内核=＃加载内核/ psf 
 learning_rate =＃您需要自己找到它，做一个对数线搜索。小比率总是会收敛，但是会缓慢。从0.4开始，每次失败均除以2。 
 maxit = 100 
 
 def损失（图像）：
返回np.sum（convolve（图像，内核）-模糊图像）
 
 def梯度（ image）：
 return convolve（convolve（图像，内核）-blurd_image）
 
去模糊= Fuzzy_image.copy（）
用于范围内的_（maxit）：
去模糊-= learning_rate * gradient（image）
  

上述方法可能是最简单的迭代反卷积算法。在实践中使用这些方法的方式是通过所谓的正则反卷积算法。这些算法通过首先指定测量图像中噪声量的函数来工作，例如 TV（I）（ I 的总变化）。然后在 L（I）+ wTV（I）上执行优化过程。如果您对此类算法感兴趣，建议阅读Amir Beck和Marc Teboulle撰写的FISTA论文。这篇论文的数学运算量很大，但是您不需要了解其中的大部分内容，只需了解如何实现电视去模糊算法即可。

除了使用正则化程序，我们使用加速方法将损耗 L（I）降至最低。内斯特罗夫加速梯度下降就是这样一个例子。有关此类方法的信息，请参阅伊曼纽尔·坎德斯（Emmanuel Candes）的布伦丹·奥多诺休（Brendan O’Donoghue）的自适应重新启动以加速渐变方案，以获取有关此类方法的信息。

如何进行图像去模糊的概述。

1. 裁剪图像这样一切都与相机保持相同的距离


2. 以与现在相同的方式查找卷积内核（首先在合成模糊图像上测试反卷积算法）


3. 实施一种迭代方法来计算去卷积蛋白


4. 去卷积图像。

I am trying to deblur an image in Python but have run into some problems. Here is what I've tried, but keep in mind that I am not an expert on this topic. According to my understanding, if you know the point spread function, you should be able to deblur the image quite simply by performing a deconvolution. However, this doesn't seem to work and I don't know if I'm doing something stupid or if I just don't understand things correctly. In Mark Newman's Computational Physics book (using Python), he touches on this subject in problem 7.9. In this problem he supplies an image that he deliberately blurred using a Gaussian point spread function (psf), and the objective of the problem is to deblur the image using a Gaussian. This is accomplished by dividing the 2D FFT of the blurred image by the 2D FFT of the psf and then taking the inverse transform. This works reasonably well.

To extend this problem, I wanted to deblur a real image taken with a camera that was deliberately out of focus. So I set up a camera and took two sets of pictures. The first set of pictures were in focus. The first was of a very small LED light in a completely darkened room and the second was of a piece of paper with text on it (using the flash). Then, without changing any of the distances or anything, I changed the focus setting on the camera so that the text was very out of focus. I then took a picture of the text using the flash and took a second picture of the LED (without the flash). Here are the blurred images.

Now, according to my understanding, the image of the blurred point light source should be the point spread function, and as such I should be able to use it to deblur my image. The problem is that when I do so I get an image that just looks like noise. After doing a little research, it seems as though noise can be a big problem when using deconvolution techniques. However, given that I have measured what I believe to be the exact point spread function, I am surprised that noise would be an issue here.

One thing I did try was to replace small values (less than epsilon) in the psf transform with either 1 or with epsilon, and I tried this with a huge range of values for epsilon. This yielded an image that was not just noise, but is also not a deblurred version of the image; it looks like a weird, blurry version of the original (non-blurred) image. Here is an image from my program (you can ignore the value of sigma, which was not used in this program).

I believe I am dealing with a noise issue, but I don't know why and I don't know what to do about it. Any advice would be much appreciated (keeping in mind that I am no expert in this area).

Note that I have deliberately not posted the code because I think that is somewhat irrelevant at this point. But I would be happy to do so if anyone thinks that would be useful. I don't think it's a programming issue because I used the same technique and it works fine when I have the known point spread function (such as when I divide the FFT of the original in-focus image by the FFT of the out-of-focus image and then inverse transform). I just don't understand why I can't seem to use my experimentally measured point spread function.

解决方案

The problem you have sought to solve is, unfortunately, more difficult than you might expect. Let me explain it in four parts. The first section assumes that you are comfortable with the Fourier transform.

1. Why you cannot solve this problem with a simple deconvolution.
2. An outline to how image deblurring can be performed.
3. Deconvolution by FFT and why it is a bad idea
4. An alternative method to perform deconvolution

But first, some notation:

I use I to represent an image and K to represent a convolution kernel. I * K is the convolution of the image I with the kernel K. F(I) is the (n-dimensional) Fourier transform of the image I and F(K) is the Fourier transform of the convolution kernel K (this is also called the point spread function, or PSF). Similarly, Fi is the inverse Fourier transform.

Why you cannot solve this problem with a simple deconvolution:

You are correct when you say that we can recover a blurred image Ib = I * K by dividing the Fourier transform of Ib by the Fourier transform of K. However, lens blur is not a convolution blurring operation. It is a modified convolution blurring operation where the blurring kernel K is dependent on the distance to the object you have photographed. Thus, the kernel changes from pixel to pixel.

You might think that this is not an issue with your image, as you have measured the correct kernel at the position of the image. However, this might not be the case, as the part of the image that is far away can influence the part of the image that is close. One way to fix this problem is to crop the image so that it is only the paper that is visible.

Why deconvolution by FFT is a bad idea:

The Convolution Theorem states that I * K = Fi(F(I)F(K)). This theorem leads to the reasonable assumption that if we have an image, Ib = I * K that is blurred by a convolution kernel K, then we can recover the deblurred image by computing I = (F(Ib)/F(K)).

Before we look at why this is a bad idea, I want to get some intuition for what the Convolution Theorem means. When we convolve an image with a kernel, then that is the same as taking the frequency components of the image and multiplying it elementwise with the frequency components of the kernel.

Now, let me explain why it is difficult to deconvolve an image with the FFT. Blurring, by default, removes high-frequency information. Thus, the high frequencies of K must go towards zero. The reason for this is that the high-frequency information of I is lost when it is blurred -- thus, the high-frequency components of Ib must go towards zero. For that to happen, the high-frequency components of K must also go towards zero.

As a result of the high-frequency components of K being almost zero, we see that the high-frequency components of Ib is amplified significantly (as we almost divide by zero) when we deconvolve with the FFT. This is not a problem in the noise-free case.

In the noisy case, however, this is a problem. The reason for this is that noise is, by definition, high-frequency information. So when we try to deconvolve Ib, the noise is amplified to an almost infinite extent. This is the reason that deconvolution by the FFT is a bad idea.

Furthermore, you need to consider how the FFT based convolution algorithm deals with boundary conditions. Normally, when we convolve images, the resolution decreases somewhat. This is unwanted behaviour, so we introduce boundary conditions that specify the pixel values of pixels outside the image. Example of such boundary conditions are

1. Pixels outside the image has the same value as the closest pixel inside the image
2. Pixels outside the image has a constant value (e.g. 0)
3. The image is part of a periodic signal, thus the row of pixel above the topmost row is equal to the bottom row of pixels.

The final boundary condition often makes sense for 1D signals. For images, however, it makes little sense. Unfortunately, the convolution theorem specifies that periodic boundary conditions are used.

In addition to this, it seems that the FFT based inversion method is significantly more sensitive to erroneous kernels than iterative methods (e.g. Gradient descent and FISTA).

An alternative method to perform deconvolution

It might seem like all hope is lost now, as all images are noisy, and deconvolving will increase the noise. However, this is not the case, as we have iterative methods to perform deconvolution. Let me start by showing you the simplest iterative method.

Let || I ||² be the squared sum of all of I's pixels. Solving the equation

Ib = I * K

with respect to I is then equivalent to solving the following optimisation problem:

min L(I) = min ||I * K - Ib||²

with respect to I. This can be done using gradient descent, as the gradient of L is given by

DL = Q * (I * K - Ib)

where Q is the kernel you get by transposing K (this is also called the matched filter in the signal processing litterature).

Thus, you can get the following iterative algorithm that will deblur an image.

from scipy.ndimage import convolve

blurred_image = # Load image
kernel = # Load kernel/psf
learning_rate = # You need to find this yourself, do a logarithmic line search. Small rate will always converge, but slowly. Start with 0.4 and divide by 2 every time it fails.
maxit = 100

def loss(image):
    return np.sum(convolve(image, kernel) - blurred_image)

def gradient(image):
    return convolve(convolve(image, kernel) - blurred_image)

deblurred = blurred_image.copy()
for _ in range(maxit):
    deblurred -= learning_rate*gradient(image)

The above method is perhaps the simplest of the iterative deconvolution algorithms. The way these are used in practice are through so-called regularised deconvolution algorithms. These algorithms work by firstly specifying a function that measures the amount of noise in an image, e.g. TV(I) (the total variation of I). Then the optimisation procedure is performed on L(I) + wTV(I). If you are interested in such algorithms, I recommend reading the FISTA paper by Amir Beck and Marc Teboulle. The paper is quite maths heavy, but you don't need to understand most of it -- only how to implement the TV deblurring algorithm.

In addition to using a regulariser, we use accelerated methods to minimise the loss L(I). One such example is Nesterov accelerated gradient descent. See Adaptive Restart for Accelerated Gradient Schemes by Brendan O'Donoghue, Emmanuel Candes for information about such methods.

An outline to how image deblurring can be performed.

1. Crop your image so that everything has same distance from the camera
2. Find the convolution kernel the same way you did now (Test your deconvolution algorithm on synthetically blurred images first)
3. Implement an iterative method to compute deconvolutoin
4. Deconvolve the image.

这篇关于图像去模糊的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！