检测网格角 [英] Detect corners of grid
问题描述
我正在尝试检测必须处理的各种图片中的网格角.图片可能会歪斜，有些可能相对正确，但是我们不能保证所有图片都会像这样.
I am trying to detect the corners of a grid within various pictures I have to process. Images can be skewed and some might be relatively well oriented but we cannot guarantee that all images will be like this.
要确定网格的角点，我尝试使用霍夫线，但无济于事.有时，霍夫线无法识别网格的边缘，因此很难确定绘制的哪些线属于网格的边缘，而哪些是网格线.
To determine the corners of the grid, I have tried using Hough lines but to no avail. Sometimes the Hough lines don't identify the edge of the grid and it is hard to determine which of the lines drawn belong to the edge of the grid and which of them are grid lines.
然后，我决定使用轮廓线来检测网格的边缘.但是，它拾取了许多轮廓，并导致了相同的问题，即确定哪些轮廓位于所标识的所有轮廓的拐角处.
I then decided to use contours to detect the edges of the grid. It however, picks up on numerous contours and it leads to the same problem of identifying which are at the corners amongst all the contours identified.
为帮助解决这个问题，我使用了双边过滤，Canny边缘检测，形态学扩张和Harris边缘检测，就像在一个与我的问题类似的问题中看到的那样.即使采取了所有这些措施，我仍然会遇到很多错误的角落，有时甚至无法识别出真正的角落.
To help aid this, I've used bilateral filtering, Canny edge detection, morphological dilation, and Harris edge detection as seen in a question with a similar problem as mine. Even after applying all those measures, I still get tons of false corners and sometimes true corners aren't identified.
我在想，是否有人可以帮助我改善拐角检测的结果，或者有人是否有完全不同的建议可以帮助解决我的问题.目的是获得优势，以便我可以使用10 X 10网格进行单应性处理，以解决图像中的倾斜问题.它还将有助于将网格正方形映射到像素空间，这非常有用.
I was wondering if anyone had a way for me to improve the results of my corner detection or if anyone had a completely different suggestion in mind that might help solve my problem. The goal is to get the corners so I can perform a homography using a 10 X 10 grid to account for skewing in the images. It will also help map grid square to pixel space which is very useful.
这是我的代码(命名和所有内容都有些草率，但稍后我会尝试修复此问题).另外，是的，我全力以赴进行双边过滤，这似乎有助于摆脱不必要的轮廓和拐角.
This is my code (a bit sloppy with the naming and all but I'll try and fix that later). Also, yes, I went all out on the bilateral filtering, it seemed to help get rid of unnecessary contours and corners.
当我尝试将Hough线应用于轮廓图像时，我似乎也遇到了一个错误:
I also seem to get one error when I tried applying the Hough lines to my contoured image:
error: (-215) img.type() == (((0) & ((1 << 3) - 1)) + (((1)-1) << 3)) in function cv::HoughLinesStandard
from PIL import Image
import numpy as np
import cv2
import glob
#import images using opencv
images = [cv2.imread(file) for file in glob.glob("SpAMImages/*.jpg")]
for image in images:
#resizes image gotten from folder and performs bilateral filtering
img = cv2.bilateralFilter(cv2.resize(image, (715,715)), 15, 800, 800)
#applies a canny edge detection filter on the images loaded from the folder
gridEdges = cv2.Canny(img, 140, 170)
#apply image dilation
kernel = np.ones((5,5), np.uint8)
gridEdges = cv2.dilate(gridEdges, kernel, iterations=1)
gridEdges = np.float32(gridEdges)
gridEdges = cv2.blur(gridEdges,(10,10))
gridEdges = cv2.cornerHarris(gridEdges,2,3,0.04)
gridEdges = cv2.dilate(gridEdges,None)
img[gridEdges>0.01*gridEdges.max()]=[0,0,255]
#draw contours on current image
imgray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
ret, thresh = cv2.threshold(imgray, 127, 255, 0)
contourImage, contours, hierarchy =
cv2.findContours(thresh,cv2.RETR_TREE,cv2.CHAIN_APPROX_SIMPLE)
contour = cv2.drawContours(img, contours, -1, (0,255,0), 3)
'''
def largest_4_sided_contour(thresh, show_contours=True):
contourImage, contours, hierarchy =
cv2.findContours(thresh,cv2.RETR_TREE,cv2.CHAIN_APPROX_SIMPLE)
contour = cv2.drawContours(img, contours, -1, (0,255,0), 3)
contours = sorted(contours, key = cv2.contourArea, reverse = True)
for cnt in contours[:min(5, len(contours))]:
print(len(cnt))
if len(cnt) == 4:
return cnt
return None
print(largest_4_sided_contour(thresh))
#applies a hough transformation to extract gridlines from the image
-----------THIS LINE BELOW GIVES ME THE ERROR-----------------
lines = cv2.HoughLines(img, 1, np.pi/180, 245)
#iterates through an array of lines gottne from the hough transform
#and draws them unto the image
for i in range(len(lines)):
for rho,theta in lines[i]:
a = np.cos(theta)
b = np.sin(theta)
x0 = a * rho
y0 = b * rho
x1 = int(x0 + 1000*(-b))
y1 = int(y0 + 1000*(a))
x2 = int(x0 - 1000*(-b))
y2 = int(y0 - 1000*(a))
cv2.line(img, (x1,y1),(x2,y2),(0,0,255),2)
cv2.imwrite('houghlines.jpg', img)
'''
#resize window because for some reason they are too large.
cv2.namedWindow('image', cv2.WINDOW_NORMAL)
cv2.resizeWindow('image', 800, 800)
#display all the images produced from above processes
cv2.imshow('image', img)
cv2.imshow('dilated edges', gridEdges)
#cv2.imshow('contour', contour)
cv2.waitKey(0)
'''
retrieve image size from imported image.
testImageWidth, testImageHeight = img.shape[:2]
print(testImageHeight, testImageWidth)'''
这些是我尝试使用轮廓检测和哈里斯角检测获得角的一些图像.
These are some images gotten from my attempt to obtain corners using contour detection and Harris corner detection.
使用轮廓和哈里斯拐角检测识别拐角:
Identifying corners using contours and harris corner detection:
还有一些我必须使用的图像示例.
And some examples of the images I have to work with.
主要示例网格:
有些偏斜的网格:
感谢您的任何帮助！
推荐答案
您正在使用OpenCV在Python中工作，但是我将为您提供使用带有DIPimage的MATLAB的答案.我希望这个答案是关于概念的，而不是关于代码的.我确信可以使用OpenCV在Python中完成所有这些工作.
You're working in Python with OpenCV, but I'm going to give you an answer using MATLAB with DIPimage. I intend this answer to be about the concepts, not about the code. I'm sure there are ways to accomplish all of these things in Python with OpenCV.
我在这里的目的是找到木板的四个角.网格本身可以猜测，因为它只是板子的等距分隔，因此无需尝试检测所有线.四个角提供了有关透视变换的所有信息.
My aim here is to find the four corners of the board. The grid itself can be guessed, since it's just an equidistant division of the board, there is no need to try to detect all lines. The four corners give all information about the perspective transformation.
检测板的最简单方法是识别板是浅色的并且具有深色背景.从灰度值图像开始，我应用一个较小的闭合(我使用了一个直径为7像素的圆，这适合于我作为示例使用的下采样图像，但是您可能想适当地增加尺寸全尺寸图片).这样得出的结果是:
The simplest way to detect the board is to recognize that it is light-colored and has a dark-colored background. Starting with a grey-value image, I apply a small closing (I used a circle with a diameter of 7 pixels, this is suitable for the down-sampled image I used as example, but you might want to increase the size appropriately for the full-size image). This gives this result:
接下来，我使用Otsu阈值选择进行二值化，并删除空洞(那部分并不重要，如果也有空洞，其余部分也可以工作).现在，我们看到的已连接组件与电路板和相邻电路板(或电路板上周围的其他白色物体)相对应.
Next, I binarize using Otsu threshold selection, and remove holes (that part is not important, the rest would work if there are holes too). The connected components that we see now correspond to the board and the neighboring boards (or whatever the other white things are around the board).
选择最大的连接组件是一个相当常见的过程.在下面的代码中，我标记图像(标识连接的组件)，计算每个连接的组件的像素数，然后选择像素最多的像素.
Selecting the largest connected component is a fairly common procedure. In the code below I label the image (identifies connected components), count the number of pixels per connected component, and select the one with the most pixels.
最后，从该结果中减去其侵蚀，只剩下板边缘的像素(这里是蓝色覆盖在输入图像上):
Finally, subtracting from this result its erosion leaves us only with the pixels at the edge of the board (here in blue overlaid on the input image):
我用来找到拐角的技巧很简单，但是在这里失败了，因为其中一个拐角不在图像中.在这四个边缘上使用Hough可能是一种更强大的解决方案.使用此其他答案以获得一些想法和代码.
The trick I'm using to find the corners is fairly simple, but fails here because one of the corners is not in the image. Using Hough on these four edges would probably be a more robust way to go about it. Use this other answer for some ideas and code on how to go about that.
无论如何，我发现最靠近图像左上角的边缘像素作为电路板的左上角.其他3个角落也是如此.这些结果是上图中的红点.
In any case, I'm finding as the top-left corner of the board the edge pixel that is closest to the top-left corner of the image. Likewise for the other 3 corners. These results are the red dots in the image above.
这里的第三个选择是将轮廓转换为多边形，使用Douglas–Peucker算法对其进行简化，丢弃沿图像边缘延伸的边缘(这是图像中没有拐角的位置)，并扩展两侧的两个边缘可以找到图像外部的顶点.
A third option here would be to convert the outline into a polygon, simplify it with the Douglas–Peucker algorithm, discard the edges that go along the image edge (this is where corners are not in the image), and extend the two edges on either side of this to find the vertex that is outside the image.
随后是MATLAB代码(带有 DIPimage ).
The MATLAB (with DIPimage) code follows.
img = readim('https://i.stack.imgur.com/GYZGa.jpg');
img = colorspace(img,'gray');
% Downsample, makes display easier
img = gaussf(img,2);
img = img(0:4:end,0:4:end);
% Simplify and binarize
sim = closing(img,7);
brd = threshold(sim); % uses Otsu threshold selection
% Fill the holes
brd = fillholes(brd);
% Keep only the largest connected component
brd = label(brd);
msr = measure(brd);
[~,I] = max(msr,'size');
brd = brd == msr(I).id;
% Extract edges
brd = brd - erosion(brd,3,'rectangular');
% Find corners
pts = findcoord(brd);
[~,top_left] = min(sum(pts.^2,2));
[~,top_right] = min(sum((pts-[imsize(brd,1),0]).^2,2));
[~,bottom_left] = min(sum((pts-[0,imsize(brd,2)]).^2,2));
[~,bottom_right] = min(sum((pts-[imsize(brd,1),imsize(brd,2)]).^2,2));
% Make an image with corner pixels set
cnr = newim(brd,'bin');
cnr(pts(top_left,1),pts(top_left,2)) = 1;
cnr(pts(top_right,1),pts(top_right,2)) = 1;
cnr(pts(bottom_left,1),pts(bottom_left,2)) = 1;
cnr(pts(bottom_right,1),pts(bottom_right,2)) = 1;
cnr = dilation(cnr,3);
% Save images
writeim(sim,'so1.png')
out = overlay(img,brd,[0,0,255]);
out = overlay(out,cnr,[255,0,0]);
writeim(out,'so2.png')
这篇关于检测网格角的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
