在进行字符识别之前使用 OpenCV 进行图像预处理(tesseract) [英] Image preprocessing with OpenCV before doing character recognition (tesseract)

问题描述

我正在尝试开发用于车牌识别的简单 PC 应用程序(Java + OpenCV + Tess4j).图像不是很好(进一步它们会很好).我想为 tesseract 预处理图像，但我一直在检测车牌(矩形检测).

我的步骤:

1) 源图像

Mat img = new Mat();img = Imgcodecs.imread("sample_photo.jpg");imgcodecs.imwrite("预处理/True_Image.png", img);

2) 灰度

Mat imgGray = new Mat();imgproc.cvtColor(img, imgGray, Imgproc.COLOR_BGR2GRAY);imgcodecs.imwrite("preprocess/Gray.png", imgGray);

3) 高斯模糊

Mat imgGaussianBlur = new Mat();Imgproc.GaussianBlur(imgGray,imgGaussianBlur,new Size(3, 3),0);imgcodecs.imwrite("预处理/gaussian_blur.png", imgGaussianBlur);

4) 自适应阈值

Mat imgAdaptiveThreshold = new Mat();imgproc.adaptiveThreshold(imgGaussianBlur, imgAdaptiveThreshold, 255, CV_ADAPTIVE_THRESH_MEAN_C ,CV_THRESH_BINARY, 99, 4);imgcodecs.imwrite("预处理/adaptive_threshold.png", imgAdaptiveThreshold);

这里应该是第 5 步，即检测板区域(现在可能甚至没有纠偏).

我使用 Paint 从图像(第 4 步之后)裁剪了所需的区域，并得到:

然后我做了 OCR(通过 tesseract、tess4j):

File imageFile = new File("preprocess/adaptive_threshold_AFTER_PAINT.png");ITesseract 实例 = new Tesseract();instance.setLanguage("eng");instance.setTessVariable("tessedit_char_whitelist", "acekopxyABCEHKMOPTXY0123456789");字符串结果 = instance.doOCR(imageFile);System.out.println(结果);

得到(足够好?)结果 - Y841ox EH"(几乎是真的)

如何在第 4 步后检测和裁剪板区域?我是否需要分 1-4 个步骤进行一些更改(改进)?希望看到一些通过 Java + OpenCV(不是 JavaCV)实现的示例.
提前致谢.

编辑(感谢@Abdul Fatir 的回答)好吧，我为那些对这个问题感兴趣的人提供了工作(至少对我来说)代码示例(Netbeans+Java+OpenCV+Tess4j).代码不是最好的，但我编写它只是为了学习.

b) 第 7 步之后的结果.绿色的是所有 minAreaRects，红色的是满足以下条件的那些:纵横比范围 (2,12) &区域范围 (300,10000)

c) 步骤 9 后的结果.选定的矩形.标准:边缘密度 > 0.5

编辑

对于边缘密度，我在上面的例子中所做的如下.

1. 将 Canny Edge 检测器直接应用于输入图像.让 cannyED 图像成为 Ic.
2. 将 Sobel 过滤器和 Ic 的结果相乘.基本上，对 Sobel 和 Canny 图像进行 AND 运算.
3. Gaussian 使用大过滤器对结果图像进行模糊处理.我使用的是 21x21.
4. 使用 OTSU 的方法对生成的图像设置阈值.你会得到一个二进制图像
5. 对于每个红色矩形，旋转该矩形内的部分(在二值图像中)使其直立.遍历矩形的像素并计算白色像素.(如何旋转?)

边缘密度 = 矩形中白色像素的数量/总数量.矩形中的像素数

6. 为边缘密度选择一个阈值.

注意:您还可以使用步骤 5 中的二值图像计算边缘密度，而不是执行步骤 1 到 3.

I'm trying to develop simple PC application for license plate recognition (Java + OpenCV + Tess4j). Images aren't really good (in further they will be good). I want to preprocess image for tesseract, and I'm stuck on detection of license plate (rectangle detection).

My steps:

1) Source Image

Mat img = new Mat();
img = Imgcodecs.imread("sample_photo.jpg"); 
Imgcodecs.imwrite("preprocess/True_Image.png", img);

2) Gray Scale

Mat imgGray = new Mat();
Imgproc.cvtColor(img, imgGray, Imgproc.COLOR_BGR2GRAY);
Imgcodecs.imwrite("preprocess/Gray.png", imgGray);

3) Gaussian Blur

Mat imgGaussianBlur = new Mat(); 
Imgproc.GaussianBlur(imgGray,imgGaussianBlur,new Size(3, 3),0);
Imgcodecs.imwrite("preprocess/gaussian_blur.png", imgGaussianBlur);  

4) Adaptive Threshold

Mat imgAdaptiveThreshold = new Mat();
Imgproc.adaptiveThreshold(imgGaussianBlur, imgAdaptiveThreshold, 255, CV_ADAPTIVE_THRESH_MEAN_C ,CV_THRESH_BINARY, 99, 4);
Imgcodecs.imwrite("preprocess/adaptive_threshold.png", imgAdaptiveThreshold);

Here should be 5th step, which is detection of plate region (probably even without deskewing for now).

I croped needed region from image (after 4th step) with Paint, and got:

Then I did OCR (via tesseract, tess4j):

File imageFile = new File("preprocess/adaptive_threshold_AFTER_PAINT.png");
ITesseract instance = new Tesseract();
instance.setLanguage("eng");
instance.setTessVariable("tessedit_char_whitelist", "acekopxyABCEHKMOPTXY0123456789");
String result = instance.doOCR(imageFile); 
System.out.println(result);

and got (good enough?) result - "Y841ox EH" (almost true)

How can I detect and crop plate region after 4th step? Have I to make some changes (improvements) in 1-4 steps? Would like to see some example implemented via Java + OpenCV (not JavaCV).
Thanks in advance.

EDIT (thanks to @Abdul Fatir's answer) Well, I provide working (for me atleast) code sample (Netbeans+Java+OpenCV+Tess4j) for those who interested in this question. Code is not the best, but I made it just for studying.
http://pastebin.com/H46wuXWn (do not forget to put tessdata folder into your project folder)

解决方案

Here's how I suggest you should do this task.

1. Convert to Grayscale.
2. Gaussian Blur with 3x3 or 5x5 filter.

3. Apply Sobel Filter to find vertical edges.

   Sobel(gray, dst, -1, 1, 0)

4. Threshold the resultant image to get a binary image.
5. Apply a morphological close operation using suitable structuring element.
6. Find contours of the resulting image.
7. Find minAreaRect of each contour. Select rectangles based on aspect ratio and minimum and maximum area.
8. For each selected contour, find edge density. Set a threshold for edge density and choose the rectangles breaching that threshold as possible plate regions.
9. Few rectangles will remain after this. You can filter them based on orientation or any criteria you deem suitable.
10. Clip these detected rectangular portions from the image after adaptiveThreshold and apply OCR.

a) Result after Step 5

b) Result after Step 7. Green ones are all the minAreaRects and the Red ones are those which satisfy the following criteria: Aspect Ratio range (2,12) & Area range (300,10000)

c) Result after Step 9. Selected rectangle. Criteria: Edge Density > 0.5

EDIT

For edge-density, what I did in the above examples is the following.

1. Apply Canny Edge detector directly to input image. Let the cannyED image be Ic.
2. Multiply results of Sobel filter and Ic. Basically, take an AND of Sobel and Canny images.
3. Gaussian Blur the resultant image with a large filter. I used 21x21.
4. Threshold the resulting image using OTSU's method. You'll get a binary image
5. For each red rectangle, rotate the portion inside this rectangle (in the binary image) to make it upright. Loop through the pixels of the rectangle and count white pixels. (How to rotate?)

Edge Density = No. of White Pixels in the Rectangle/Total no. of Pixels in the rectangle

6. Choose a threshold for edge density.

NOTE: Instead of going through steps 1 to 3, you can also use the binary image from step 5 for calculating edge density.

这篇关于在进行字符识别之前使用 OpenCV 进行图像预处理(tesseract)的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！