用于识别2D特征的图像处理 [英] Image Processing for recognizing 2D features

问题描述

我创建了一个iPhone应用程序，可以扫描一张方格纸的图像，然后可以告诉我哪些方块已被涂黑，哪些方块是空白的。

I've created an iPhone app that can scan an image of a page of graph paper and can then tell me which squares have been blacked out and which squares are blank.

我这样做是通过从左到右扫描并使用方格纸的线作为指南。当我遇到方格纸线时，我开始寻找黑色，直到我再次点击方格纸线。然后，我继续沿扫描线继续前进，完全扫描方块为黑色。然后我继续下一个方框。在这一行的最后，我在新行上开始扫描之前跳过这么多像素（因为我已经知道每个盒子有多高）。

I do this by scanning from left to right and use the graph paper's lines as guides. When I encounter a graph paper line, I start to look for black, until I hit the graph paper line again. Then, instead of continuing along the scan line, I go ahead and completely scan the square for black. Then I continue on to the next box. At the end of the line, I skip down so many pixels before starting the scan on a new line (since I have already figured out how tall each box is).

这种作品，但有问题。有时我把图形线误认为是黑色。有时候，如果图像偏斜，或者我的页面没有均匀的照明，那么我的效果就不会很好。

This sort of works, but there are problems. Sometimes I mistake the graph lines as "black". Sometimes, if the image is skewed, or I don't have uniform lighting across the page, then I don't get good results.

我想要的是什么做的是指定一些对齐框，然后我调整大小并旋转（和倾斜）图片以与那些对齐。然后，我想，一旦我将图像对齐，我就会知道所有盒子的位置，并且不必扫描盒子，只需扫描盒子的位置，看看它们是否是黑色的。这应该更快，更可靠。如果我操作来自相机的图像，我可以更灵活地要求用户对齐图片以匹配对齐标记，而不是自己对齐图像。

What I'd like to do is to specify a few "alignment" boxes that I then resize and rotate (and skew) the picture to align with those. Then, I was thinking that once I have the image aligned, I would then know where all the boxes are and won't have to scan for the boxes, just scan inside the location of the boxes to see if they are black. This should be faster and more reliable. And if I were to operate on images coming from the camera, I'd have more flexibility in asking the user to align the picture to match the alignment marks, rather than having to align the image myself.

鉴于这是我的第一个图像处理项目，我觉得我正在重新发明轮子。我想知道如何做到这一点，以及是否利用像OpenCV这样的库。

Given that this is my first Image Processing project, I feel like I am reinventing the wheel. I'd like suggestions on how to do this, and whether to utilize libraries like OpenCV.

我附上的图像类似于我想要处理的图像。我正在寻找具有大量黑色标记的所有正方形的列表，即A8，C4，E7，G4，H1，J9。

I am enclosing an image similar to what I would like processed. I am looking for a list of all squares that have a significant amount of black marking, i.e. A8, C4, E7, G4, H1, J9.

问题要注意：

• 图像的光线覆盖范围可能并不理想，但在整个图像中应该相对一致（即没有阴影）


• 所有方块都可能为空或全黑，算法需要能够确定


• 图像可能会偏斜或旋转任何方块的轴。绕z轴的旋转可能很容易修复。可以围绕x轴或y轴旋转，使得图像的一侧比另一侧宽。但是，如果我实时扫描图像来自相机，我可以要求用户将对齐标记与屏幕上的标记对齐。如何最好地确保对齐以便为用户提供适当的反馈？只需检查以确保4个角落是暗的，当相机指向黑色表面时可能会导致误报。


• 并非每个方块都会同样或持续变黑，但我我认为会有足够的黑色使人眼无可置疑。


• 蓝色网格可能有用，但有时黑色标记可能与蓝色网格重叠。我认为虚拟网格可能比依赖打印网格更好。我认为使用对齐标记来对齐图像，然后可以布置精确的虚拟网格。然后可以对每个网格框的内容进行采样，以查看它是否主要是黑色，而不是从左到右扫描，不是吗？这是另一个在网格上有更多标记的图像。在此图像中，除了A8，C4，E7，G4，H1，J9中的先前标记外，我还标记了E2，G8和G9，以及I4和J4，您可以看到蓝色网格是如何被遮挡的。

• Light coverage of the image may not be ideal, but should be relatively consistent across the image (i.e. no shadows)
• All squares may be empty or all dark, and the algorithm needs to be able to determine that
• the image may be skewed or rotated about any of the axis. Rotation about the z axis maybe easy to fix. There may be rotation around the x or y axis making ones side of the image be wider than the other. However, if I scan the image in realtime as it comes from the camera, I can ask the user to align the alignment marks with marks on the screen. How best to ensure that alignment to give the user appropriate feedback? Just checking to make sure that the 4 corners are dark could result in a false positive when the camera is pointing to a black surface.
• not every square will be equally or consistently blacked, but I think there will be enough black to make it unquestionable to a human eye.
• the blue grid may be useful, but there are cases where the black markings may overlap the blue grid. I think a virtual grid is probably better than relying on the printed grid. I would think that using the alignment markers to align the image, would then allow for a precise virtual grid to be laid out. And then the contents of each grid box could be sampled, to see if it was predominantly black, vs scanning from left-to-right, no? Here is another image with more markings on the grid. In this image, in addition to the previous marking in A8, C4, E7, G4, H1, J9, I have marked E2, G8 and G9, and I4 and J4 and you can see how the blue grid is obscured.

• 这是我这个项目的第一阶段。最后我想扩展这个算法，以便能够处理至少几百个插槽和可能不同的颜色。

推荐答案

首先，这个问题让我想起了一些可能对我们学习有用的演示：

To start with, this problem reminded me a bit of these demo's that might be useful to learn from:

• DNA微阵列图像处理


• Matlab Sudoku求解器


• Iphone Sudoku解算器博客文章，解释图像处理

• The DNA microarray image processing
• The Matlab Sudoku solver
• The Iphone Sudoku solver blog post, explaining the image processing

就我个人而言，我认为最简单的方法是检测图像中的方块。

Personally, I think the most simple approach would be to detect the squares in your image.

1）删除背景和小瑕疵

1) Remove the background and small cruft

f_makebw = @(I) im2bw(I.data, double(median(I.data(:)))/1.3);
bw = ~blockproc(im, [128 128], f_makebw);
bw = bwareaopen(bw, 30);

2）删除除正方形和圆形之外的所有内容。

2) Remove everything but the squares and circles.

se = strel('disk', 5);
bw = imerode(bw, se);

% Detect the squares and cricles via morphology
[B, L] = bwboundaries(bw, 'noholes');

3）使用来自 regionprops 。 范围度量标准衡量填充边界框的比例。这使得它成为区分圆形和正方形的
不错的衡量标准

3) Detect the squares using 'extend' from regionprops. The 'Extent' metric measures what proportion of the bounding-box is filled. This makes it a nice measure to distinguish between circles and squares

stats = regionprops(L, 'Extent'); 
extent = [stats.Extent];
idx1 = find(extent > 0.8);
bw = ismember(L, idx1);

4）这将为您提供功能，同步或纠正图像。一种简单而强大的方法是通过自相关函数。

4) This leaves you with your features, to synchronize or rectify the image with. An easy, and robust way, to do this, is via the Autocorrelation Function.

这样可以很容易地检测出漂亮的峰值。可以通过匈牙利算法将这些峰与来自模板图像的ACF峰匹配。匹配后，您可以校正旋转和缩放，因为您现在拥有一个可以解决的线性系统：

This gives nice peaks, which are easily detected. These peaks can be matched against the ACF peaks from a template image via the Hungarian algorithm. Once matched, you can correct rotation and scaling as you now have a linear system which you can solve:

x = Ax'

然后可以使用普通的交叉相关对相同的预定义模板进行翻译。

Translation can then be corrected using run-of-the-mill cross correlation against the same pre defined template.

如果一切顺利，你知道有一个对齐或同步的图像，这应该有助于确定点的位置。

If all goes well, you know have an aligned or synchronized image, which should help considerably in determining the position of the dots.

这篇关于用于识别2D特征的图像处理的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！