分割车牌字符 [英] Segmenting License Plate Characters

问题描述

我在从车牌图像中分割字符时遇到问题.我已经应用了以下方法来提取车牌字符"

1. 车牌图像的自适应阈值.
2. 选择具有特定纵横比的轮廓.

如果附加文件中的车牌图像中有任何阴影，由于二值化不当，我无法正确分割字符.图像中的阴影合并图像中的相邻字符.

我对具有不同窗口大小的图像进行了阈值处理.结果附后.如果图像中有阴影，如何从图像中分割字符?我正在使用 OpenCV.

我在 OpenCV 中使用了以下函数来对我的车牌图像进行阈值处理:

cvAdaptiveThreshold(licensePlateImg, threshImg, 255, CV_ADAPTIVE_THRESH_GAUSSIAN_C, CV_THRESH_BINARY_INV, wind);

我尝试过不同的窗口大小(wind)和不同的adaptiveMethod(ADAPTIVE_THRESH_MEAN_C 和 ADAPTIVE_THRESH_GAUSSIAN_C)获取阈值图像.

解决方案

在我开始之前，我知道您正在寻求在 OpenCV C++ 中实现此算法，但我的算法需要 FFT 和 numpy/scipy 包非常棒.因此，我将在 OpenCV 中使用 Python 为您提供该算法的实现.该代码实际上与 C++ API 非常相似，您可以轻松地将其转录过来.这样，它最大限度地减少了我学习(或者更确切地说是重新学习......)API所需的时间，我宁愿给你算法和我为执行这项任务所做的步骤，以免浪费任何时间.

因此，我将向您概述我将要做什么.然后，我将向您展示使用 numpy、scipy 和 OpenCV 包的 Python 代码.作为对使用 MATLAB 的人的奖励，我将向您展示 MATLAB 等效程序，其中包含可启动的 MATLAB 代码！

<小时>

您可以做的是尝试使用同态过滤.在基本术语中，我们可以根据照度和反射率的乘积来表示图像.假定照明缓慢变化并且是动态范围的主要贡献者.这本质上是低频内容.反射率代表物体的细节，并假设变化迅速.这也是局部对比度的主要贡献者，本质上是高频内容.

图像可以表示为这两者的产品.同态过滤尝试并拆分这些组件，然后我们单独过滤它们.完成后，我们将结果合并在一起.由于这是一个乘法模型，习惯上使用 log 运算，以便我们可以将乘积表示为两个项的总和.这两个项被单独过滤，缩放以强调或不强调它们对图像的贡献，求和，然后取反对数.

阴影是由光照引起的，所以我们可以做的是减少这种阴影对图像的影响.我们还可以提高反射率，这样我们可以获得更好的边缘，因为边缘与高频信息相关.

我们通常使用低通滤波器过滤照明，而使用高通滤波器过滤反射.在这种情况下，我将选择 sigma 为 10 的高斯内核作为低通滤波器.取1与低通滤波器相减即可得到高通滤波器.我将图像转换到对数域，然后使用低通和高通滤波器在频域中过滤图像.然后我缩放低通和高通结果，将这些组件添加回来，然后取反对数.该图像现在更适合进行阈值处理，因为图像的变化很小.

我做的额外后处理是对图像进行阈值处理.字母比整体背景更暗，因此任何低于某个阈值的像素都将被归类为文本.我选择了强度为 65 的阈值.在此之后，我还清除了接触边界的所有像素，然后删除图像中总面积小于 160 (MATLAB) 或 120 (Python) 像素的任何区域.我还裁剪了图像的一些列，因为我们的分析不需要它们.

<小时>

这里有几个注意事项:

警告 #1 - 删除边框

移除任何接触边框的像素没有内置于 OpenCV 中.但是，MATLAB 有一个等效项，称为 imclearborder.我将在我的 MATLAB 代码中使用它，但对于 OpenCV，这是以下算法:

• 找出图像中的所有轮廓
• 对于图像中的每个轮廓，检查是否有任何轮廓像素在图像的边界内
• 如果有，请标记要移除的轮廓
• 对于我们要删除的每个轮廓，只需将整个轮廓绘制为黑色

我在我的代码中创建了一个名为 imclearborder(imgBW, radius) 的方法，其中 radius 是您想要清除的边框内的多少像素.>

警告 #2 - 删除特定区域下方的像素区域

删除小于一定数量的任何区域也未在 OpenCV 中实现.在 MATLAB 中，这可以使用 bwareaopen 方便地给出.其基本算法是:

• 找出图像中的所有轮廓
• 如果要填充内部，请分析每个轮廓的区域填充了多少
• 任何小于一定数量的区域，通过用黑色填充内部来清除此轮廓

我创建了一个名为 bwareaopen(imgBW) 的方法来为我们执行此操作.

注意事项 #3 - 用于移除像素区域的区域参数

对于 Python 代码，我不得不玩弄这个参数，我选择了 120.160 用于 MATLAB.对于python，120去掉了一些不需要的字符.我猜我的 bwareaopen 实现与 MATLAB 相比有所不同，这可能就是我得到不同结果的原因.

<小时>

不用多说，这是代码.请注意，我没有使用空间过滤.您可以在 OpenCV 中使用 filter2D 并将此图像与高斯内核卷积，但我没有这样做，因为在使用低通和高通滤波器时，同态滤波传统上是在频域中完成的.您可以使用空间过滤对此进行探索，但您还必须事先知道内核的大小.使用频域滤波，您只需要知道滤波器的标准偏差，这只是一个参数对比两个参数.

此外，对于 Python 代码，我将您的图像下载到我的计算机上并运行了脚本.对于 MATLAB，您可以在使用图像处理工具箱读取图像时直接引用图像的超链接.

<小时>

Python 代码

import cv2 # 用于 OpenCV 模块(用于图像 I/O 和轮廓查找)import numpy as np # 用于通用数组操作导入 scipy.fftpack # 用于 FFT2#### imclearborder 定义def imclearborder(imgBW, 半径):# 给定一个黑白图像，首先找到它的所有轮廓imgBWcopy = imgBW.copy()轮廓，层次结构 = cv2.findContours(imgBWcopy.copy(), cv2.RETR_LIST,cv2.CHAIN_APPROX_SIMPLE)# 获取图像的尺寸imgRows = imgBW.shape[0]imgCols = imgBW.shape[1]contourList = [] # 接触边界的轮廓的 ID 列表# 对于每个轮廓...对于 np.arange(len(contours)) 中的 idx:# 获取第 i 个轮廓cnt = 轮廓[idx]# 查看轮廓中的每个点对于 cnt 中的 pt:rowCnt = pt[0][1]colCnt = pt[0][0]# 如果这是在边界的半径内# 这个轮廓再见！check1 = (rowCnt >= 0 and rowCnt < radius) or (rowCnt >= imgRows-1-radius and rowCnt < imgRows)check2 = (colCnt >= 0 and colCnt < radius) or (colCnt >= imgCols-1-radius and colCnt < imgCols)如果检查1或检查2:轮廓列表.append(idx)休息对于轮廓列表中的 idx:cv2.drawContours(imgBWcopy, 轮廓, idx, (0,0,0), -1)返回 imgBWcopy#### bwareaopen 定义def bwareaopen(imgBW, areaPixels):# 给定一个黑白图像，首先找到它的所有轮廓imgBWcopy = imgBW.copy()轮廓，层次结构 = cv2.findContours(imgBWcopy.copy(), cv2.RETR_LIST,cv2.CHAIN_APPROX_SIMPLE)# 对于每个轮廓，确定其总占据面积对于 np.arange(len(contours)) 中的 idx:area = cv2.contourArea(contours[idx])if (area >= 0 and area <= areaPixels):cv2.drawContours(imgBWcopy, 轮廓, idx, (0,0,0), -1)返回 imgBWcopy####主程序# 读入图像img = cv2.imread('5DnwY.jpg', 0)# 行数和列数行 = img.shape[0]cols = img.shape[1]# 去掉开头和结尾的一些列img = img[:, 59:cols-20]# 行数和列数行 = img.shape[0]cols = img.shape[1]# 将图像转换为 0 到 1，然后执行 log(1 + I)imgLog = np.log1p(np.array(img, dtype="float")/255)# 创建 sigma = 10 的高斯掩码M = 2*行 + 1N = 2*cols + 1西格玛 = 10(X,Y) = np.meshgrid(np.linspace(0,N-1,N), np.linspace(0,M-1,M))centerX = np.ceil(N/2)centerY = np.ceil(M/2)gaussianNumerator = (X - centerX)**2 + (Y - centerY)**2# 低通和高通滤波器Hlow = np.exp(-gaussianNumerator/(2*sigma*sigma))高 = 1 - 低# 移动过滤器的原点，使其位于左上角# 匹配输入图像HlowShift = scipy.fftpack.ifftshift(Hlow.copy())HhighShift = scipy.fftpack.ifftshift(Hhigh.copy())# 过滤图像并裁剪如果 = scipy.fftpack.fft2(imgLog.copy(), (M,N))Ioutlow = scipy.real(scipy.fftpack.ifft2(If.copy() * HlowShift, (M,N)))Iouthigh = scipy.real(scipy.fftpack.ifft2(If.copy() * HhighShift, (M,N)))# 设置缩放因子并添加伽马1 = 0.3伽马2 = 1.5Iout = gamma1*Ioutlow[0:rows,0:cols] + gamma2*Iouthigh[0:rows,0:cols]# Anti-log 然后重新缩放到 [0,1]Ihmf = np.expm1(Iout)Ihmf = (Ihmf - np.min(Ihmf))/(np.max(Ihmf) - np.min(Ihmf))Ihmf2 = np.array(255*Ihmf, dtype="uint8")# 图像阈值 - 强度低于 65 的任何内容都设置为白色Ithresh = Ihmf2 <65Ithresh = 255*Ithresh.astype("uint8")# 清除边界.选择 5 像素的边框半径Iclear = imclearborder(Ithresh, 5)# 消除面积低于 120 像素的区域Iopen = bwareaopen(Iclear, 120)# 显示所有图像cv2.imshow('原图', img)cv2.imshow('同态滤波结果', Ihmf2)cv2.imshow('阈值结果', Ithresh)cv2.imshow('打开结果', Iopen)cv2.waitKey(0)cv2.destroyAllWindows()

<小时>

MATLAB 代码

清除所有；关闭所有;% 读入图像I = imread('http://i.stack.imgur.com/5DnwY.jpg');% 从开头和结尾删除一些列I = I(:,60:end-20);% 投射到双倍并做日志.我们添加 1 以避免 log(0) 错误.我 = im2double(I);I = log(1 + I);% 在频域中创建高斯掩码% 我们必须指定我们的掩码为图像大小的两倍以避免% 混叠.M = 2*大小(I,1) + 1;N = 2*size(I,2) + 1;西格玛 = 10;[X, Y] = 网格网格(1:N,1:M);centerX = ceil(N/2);centerY = ceil(M/2);gaussianNumerator = (X - centerX).^2 + (Y - centerY).^2;% 低通和高通滤波器Hlow = exp(-gaussianNumerator./(2*sigma.^2));高 = 1 - 低；% 移动过滤器的原点，使其位于左上角以匹配% 输入图像Hlow = ifftshift(Hlow);Hhigh = ifftshift(Hhigh);% 过滤图像，并裁剪如果 = fft2(I, M, N);Ioutlow = real(ifft2(Hlow .* If));Iouthigh = real(ifft2(Hhigh .* If));% 设置缩放因子然后添加伽马1 = 0.3；伽马2 = 1.5；Iout = gamma1*Ioutlow(1:size(I,1),1:size(I,2)) + ...gamma2*Iouthigh(1:size(I,1),1:size(I,2));% Anti-log 然后重新缩放到 [0,1]Ihmf = exp(Iout) - 1;Ihmf = (Ihmf - min(Ihmf(:)))/(max(Ihmf(:)) - min(Ihmf(:)));% 图像阈值 - 强度低于 65 的任何内容都设置为白色Ithresh = Ihmf <65/255；% 去除边框像素Iclear = imclearborder(Ithresh, 8);% 消除面积低于 160 像素的区域Iopen = bwareaopen(Iclear, 160);% 显示所有结果数字;子图(4,1,1)；显示(一)；title('原图');子图(4,1,2)；imshow(Ihmf);title('同态过滤结果');子图(4,1,3)；imshow(Ithresh);title('阈值结果');子图(4,1,4)；imshow(Iopen);title('打开结果');

<小时>

这是我得到的结果:

Python

请注意，我重新排列了窗口，使它们在单列中对齐.

MATLAB

I am facing a problem in segmenting characters from a license plate image. I have applied following method to extract license plate characters"

1. Adaptive threshold the license plate image.
2. Select contours which having particular aspect ratio.

If there is any shade in the license plate image as in attached file, I am not able to properly segment the characters due to improper binarization. The shade in the image merges adjacent characters in the image.

I have thresholded the images with different window sizes. The results are attached. How can I segment characters from image if there is shade in the image? I am using OpenCV.

I have used following function in OpenCV to threshold my license plate image:

cvAdaptiveThreshold(licensePlateImg, threshImg, 255, CV_ADAPTIVE_THRESH_GAUSSIAN_C, CV_THRESH_BINARY_INV, wind);

I have tried with different window sizes (wind) and different adaptiveMethod (ADAPTIVE_THRESH_MEAN_C and ADAPTIVE_THRESH_GAUSSIAN_C) to get the thresholded images.

解决方案

Before I start, I know you are seeking an implementation of this algorithm in OpenCV C++, but my algorithm requires the FFT and the numpy / scipy packages are awesome for that. As such, I will give you an implementation of the algorithm in OpenCV using Python instead. The code is actually quite similar to the C++ API that you can easily transcribe that over instead. That way, it minimizes the amount of time it will take for me to learn (or rather relearn...) the API and I would rather give you the algorithm and the steps I did to perform this task to not waste any time at all.

As such, I will give you a general overview of what I would do. I will then show you Python code that uses numpy, scipy and the OpenCV packages. As a bonus for those who use MATLAB, I will show you the MATLAB equivalent, with MATLAB code to boot!

---

What you can do is try to use homomorphic filtering. In basic terms, we can represent an image in terms of a product of illumination and reflectance. Illumination is assumed to be slowly varying and the main contributor of dynamic range. This is essentially low frequency content. Reflectance represents details of objects and assumed to vary rapidly. This is also the primary contributor to local contrast and is essentially high frequency content.

The image can be represented as a product of these two. Homomorphic filtering tries and splits up these components and we filter them individually. We then combine the results together when we are finished. As this is a multiplicative model, it's customary to use a log operation so that we can express the product as a sum of two terms. These two terms are filtered individually, scaled to emphasize or de-emphasize their contributions to the image, summed, then the anti-log is taken.

The shading is due to the illumination, and so what we can do is decrease the contribution that this shading does over the image. We can also boost the reflectance so we can get some better edges as edges are associated with high frequency information.

We usually filter the illumination using a low-pass filter, while the reflectance with a high-pass filter. In this case, I'm going to choose a Gaussian kernel with a sigma of 10 as the low-pass filter. A high-pass filter can be obtained by taking 1 and subtracting with the low-pass filter. I transform the image into the log domain, then filter the image in the frequency domain using the low-pass and high-pass filters. I then scale the low pass and high pass results, add these components back, then take the anti-log. This image is now better suited to be thresholded as the image has low variation.

What I do as additional post-processing is that I threshold the image. The letters are darker than the overall background, so any pixels that are lower than a certain threshold would be classified as text. I chose the threshold to be intensity 65. After this, I also clear off any pixels that are touching the border, then remove any areas of the image that have less than 160 (MATLAB) or 120 (Python) pixels of total area. I also crop out some of the columns of the image as they are not needed for our analysis.

---

Here are a couple of caveats for you:

Caveat #1 - Removing borders

Removing any pixels that touch the border is not built into OpenCV. However, MATLAB has an equivalent called imclearborder. I'll use this in my MATLAB code, but for OpenCV, this was the following algorithm:

• Find all of the contours in the image
• For each contour that is in the image, check to see if any of the contour pixels are within the border of the image
• If any are, mark this contour for removal
• For each contour we want to remove, simply draw this whole contour in black

I created a method called imclearborder(imgBW, radius) in my code, where radius is how many pixels within the border you want to clear stuff up.

Caveat #2 - Removing pixel areas below a certain area

Removing any areas where they are less than a certain amount is also not implemented in OpenCV. In MATLAB, this is conveniently given using bwareaopen. The basic algorithm for this is:

• Find all of the contours in the image
• Analyze how much each contour's area fills up if you were to fill in the interior
• Any areas that are less than a certain amount, clear this contour by filling the interior with black

I created a method called bwareaopen(imgBW) that does this for us.

Caveat #3 - Area parameter for removing pixel areas

For the Python code, I had to play around with this parameter and I settled for 120. 160 was used for MATLAB. For python, 120 got rid of some of the characters, which is not desired. I'm guessing my implementation of bwareaopen in comparison to MATLAB's is different, which is probably why I'm getting different results.

---

Without further ado, here's the code. Take note that I did not use spatial filtering. You could use filter2D in OpenCV and convolve this image with the Gaussian kernel, but I did not do that as Homomorphic Filtering when using low-pass and high-pass filters are traditionally done in the frequency domain. You could explore this using spatial filtering, but you would also have to know the size of your kernels before hand. With frequency domain filtering, you just need to know the standard deviation of the filter, and that's just one parameter in comparison to two.

Also, for the Python code, I downloaded your image on to my computer and ran the script. For MATLAB, you can directly reference the hyperlink to the image when reading it in with the Image Processing toolbox.

---

Python code

import cv2 # For OpenCV modules (For Image I/O and Contour Finding)
import numpy as np # For general purpose array manipulation
import scipy.fftpack # For FFT2 

#### imclearborder definition

def imclearborder(imgBW, radius):

    # Given a black and white image, first find all of its contours
    imgBWcopy = imgBW.copy()
    contours,hierarchy = cv2.findContours(imgBWcopy.copy(), cv2.RETR_LIST, 
        cv2.CHAIN_APPROX_SIMPLE)

    # Get dimensions of image
    imgRows = imgBW.shape[0]
    imgCols = imgBW.shape[1]    

    contourList = [] # ID list of contours that touch the border

    # For each contour...
    for idx in np.arange(len(contours)):
        # Get the i'th contour
        cnt = contours[idx]

        # Look at each point in the contour
        for pt in cnt:
            rowCnt = pt[0][1]
            colCnt = pt[0][0]

            # If this is within the radius of the border
            # this contour goes bye bye!
            check1 = (rowCnt >= 0 and rowCnt < radius) or (rowCnt >= imgRows-1-radius and rowCnt < imgRows)
            check2 = (colCnt >= 0 and colCnt < radius) or (colCnt >= imgCols-1-radius and colCnt < imgCols)

            if check1 or check2:
                contourList.append(idx)
                break

    for idx in contourList:
        cv2.drawContours(imgBWcopy, contours, idx, (0,0,0), -1)

    return imgBWcopy

#### bwareaopen definition
def bwareaopen(imgBW, areaPixels):
    # Given a black and white image, first find all of its contours
    imgBWcopy = imgBW.copy()
    contours,hierarchy = cv2.findContours(imgBWcopy.copy(), cv2.RETR_LIST, 
        cv2.CHAIN_APPROX_SIMPLE)

    # For each contour, determine its total occupying area
    for idx in np.arange(len(contours)):
        area = cv2.contourArea(contours[idx])
        if (area >= 0 and area <= areaPixels):
            cv2.drawContours(imgBWcopy, contours, idx, (0,0,0), -1)

    return imgBWcopy

#### Main program

# Read in image
img = cv2.imread('5DnwY.jpg', 0)

# Number of rows and columns
rows = img.shape[0]
cols = img.shape[1]

# Remove some columns from the beginning and end
img = img[:, 59:cols-20]

# Number of rows and columns
rows = img.shape[0]
cols = img.shape[1]

# Convert image to 0 to 1, then do log(1 + I)
imgLog = np.log1p(np.array(img, dtype="float") / 255)

# Create Gaussian mask of sigma = 10
M = 2*rows + 1
N = 2*cols + 1
sigma = 10
(X,Y) = np.meshgrid(np.linspace(0,N-1,N), np.linspace(0,M-1,M))
centerX = np.ceil(N/2)
centerY = np.ceil(M/2)
gaussianNumerator = (X - centerX)**2 + (Y - centerY)**2

# Low pass and high pass filters
Hlow = np.exp(-gaussianNumerator / (2*sigma*sigma))
Hhigh = 1 - Hlow

# Move origin of filters so that it's at the top left corner to
# match with the input image
HlowShift = scipy.fftpack.ifftshift(Hlow.copy())
HhighShift = scipy.fftpack.ifftshift(Hhigh.copy())

# Filter the image and crop
If = scipy.fftpack.fft2(imgLog.copy(), (M,N))
Ioutlow = scipy.real(scipy.fftpack.ifft2(If.copy() * HlowShift, (M,N)))
Iouthigh = scipy.real(scipy.fftpack.ifft2(If.copy() * HhighShift, (M,N)))

# Set scaling factors and add
gamma1 = 0.3
gamma2 = 1.5
Iout = gamma1*Ioutlow[0:rows,0:cols] + gamma2*Iouthigh[0:rows,0:cols]

# Anti-log then rescale to [0,1]
Ihmf = np.expm1(Iout)
Ihmf = (Ihmf - np.min(Ihmf)) / (np.max(Ihmf) - np.min(Ihmf))
Ihmf2 = np.array(255*Ihmf, dtype="uint8")

# Threshold the image - Anything below intensity 65 gets set to white
Ithresh = Ihmf2 < 65
Ithresh = 255*Ithresh.astype("uint8")

# Clear off the border.  Choose a border radius of 5 pixels
Iclear = imclearborder(Ithresh, 5)

# Eliminate regions that have areas below 120 pixels
Iopen = bwareaopen(Iclear, 120)

# Show all images
cv2.imshow('Original Image', img)
cv2.imshow('Homomorphic Filtered Result', Ihmf2)
cv2.imshow('Thresholded Result', Ithresh)
cv2.imshow('Opened Result', Iopen)
cv2.waitKey(0)
cv2.destroyAllWindows()

---

MATLAB code

clear all;
close all;

% Read in image
I = imread('http://i.stack.imgur.com/5DnwY.jpg');

% Remove some columns from the beginning and end
I = I(:,60:end-20);

% Cast to double and do log.  We add with 1 to avoid log(0) error.
I = im2double(I);
I = log(1 + I);

% Create Gaussian mask in frequency domain
% We must specify our mask to be twice the size of the image to avoid
% aliasing.
M = 2*size(I,1) + 1;
N = 2*size(I,2) + 1;
sigma = 10;
[X, Y] = meshgrid(1:N,1:M);
centerX = ceil(N/2);
centerY = ceil(M/2);
gaussianNumerator = (X - centerX).^2 + (Y - centerY).^2;

% Low pass and high pass filters
Hlow = exp(-gaussianNumerator./(2*sigma.^2));
Hhigh = 1 - Hlow;

% Move origin of filters so that it's at the top left corner to match with
% input image
Hlow = ifftshift(Hlow);
Hhigh = ifftshift(Hhigh);

% Filter the image, and crop
If = fft2(I, M, N);
Ioutlow = real(ifft2(Hlow .* If));
Iouthigh = real(ifft2(Hhigh .* If));

% Set scaling factors then add
gamma1 = 0.3;
gamma2 = 1.5;
Iout = gamma1*Ioutlow(1:size(I,1),1:size(I,2)) + ...
       gamma2*Iouthigh(1:size(I,1),1:size(I,2));

% Anti-log then rescale to [0,1]
Ihmf = exp(Iout) - 1;
Ihmf = (Ihmf - min(Ihmf(:))) / (max(Ihmf(:)) - min(Ihmf(:)));

% Threshold the image - Anything below intensity 65 gets set to white
Ithresh = Ihmf < 65/255;

% Remove border pixels
Iclear = imclearborder(Ithresh, 8);

% Eliminate regions that have areas below 160 pixels
Iopen = bwareaopen(Iclear, 160);

% Show all of the results
figure;
subplot(4,1,1);
imshow(I);
title('Original Image');
subplot(4,1,2);
imshow(Ihmf);
title('Homomorphic Filtered Result');
subplot(4,1,3);
imshow(Ithresh);
title('Thresholded Result');
subplot(4,1,4);
imshow(Iopen);
title('Opened Result');

---

This is the result I get:

Python

Take note that I re-arranged the windows so that they're aligned in a single column.

MATLAB

这篇关于分割车牌字符的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！