检测带圆角的卡片边缘 [英] Detecting edges of a card with rounded corners
问题描述
您好我目前正在使用OCR阅读应用程序，我已成功使用AVFoundation框架捕获卡片图像。
Hi currently i am working on an OCR reading app where i have successfully able to capture the card image by using AVFoundation framework.
下一步，我需要找出卡片的边缘，这样我就可以从主要拍摄的图像中裁剪出卡片图像。之后我可以将它发送到OCR引擎进行处理。
For next step, i need to find out edges of the card , so that i can crop the card image from main captured image & later i can sent it to OCR engine for processing.
现在的主要问题是找到卡片的边缘。我使用下面的代码（取自另一个开源项目）使用OpenCV用于此目的。如果卡是纯矩形卡或纸，它工作正常。但是当我使用带圆角的卡（例如驾驶执照）时，它无法检测到。另外我在OpenCV上没有太多的专业知识，任何人都可以帮助我解决这个问题吗？
The main problem is now to find the edges of the card & i am using below code(taken from another open source project) which uses OpenCV for this purpose.It is working fine if the card is pure rectangular Card or Paper. But when i use a card with rounded corner (e.g Driving License), it is failed to detect . Also i dont have much expertise in OpenCV , Can any one help me in solving this issue?
- (void)detectEdges
{
cv::Mat original = [MAOpenCV cvMatFromUIImage:_adjustedImage];
CGSize targetSize = _sourceImageView.contentSize;
cv::resize(original, original, cvSize(targetSize.width, targetSize.height));
cv::vector<cv::vector<cv::Point>>squares;
cv::vector<cv::Point> largest_square;
find_squares(original, squares);
find_largest_square(squares, largest_square);
if (largest_square.size() == 4)
{
// Manually sorting points, needs major improvement. Sorry.
NSMutableArray *points = [NSMutableArray array];
NSMutableDictionary *sortedPoints = [NSMutableDictionary dictionary];
for (int i = 0; i < 4; i++)
{
NSDictionary *dict = [NSDictionary dictionaryWithObjectsAndKeys:[NSValue valueWithCGPoint:CGPointMake(largest_square[i].x, largest_square[i].y)], @"point" , [NSNumber numberWithInt:(largest_square[i].x + largest_square[i].y)], @"value", nil];
[points addObject:dict];
}
int min = [[points valueForKeyPath:@"@min.value"] intValue];
int max = [[points valueForKeyPath:@"@max.value"] intValue];
int minIndex;
int maxIndex;
int missingIndexOne;
int missingIndexTwo;
for (int i = 0; i < 4; i++)
{
NSDictionary *dict = [points objectAtIndex:i];
if ([[dict objectForKey:@"value"] intValue] == min)
{
[sortedPoints setObject:[dict objectForKey:@"point"] forKey:@"0"];
minIndex = i;
continue;
}
if ([[dict objectForKey:@"value"] intValue] == max)
{
[sortedPoints setObject:[dict objectForKey:@"point"] forKey:@"2"];
maxIndex = i;
continue;
}
NSLog(@"MSSSING %i", i);
missingIndexOne = i;
}
for (int i = 0; i < 4; i++)
{
if (missingIndexOne != i && minIndex != i && maxIndex != i)
{
missingIndexTwo = i;
}
}
if (largest_square[missingIndexOne].x < largest_square[missingIndexTwo].x)
{
//2nd Point Found
[sortedPoints setObject:[[points objectAtIndex:missingIndexOne] objectForKey:@"point"] forKey:@"3"];
[sortedPoints setObject:[[points objectAtIndex:missingIndexTwo] objectForKey:@"point"] forKey:@"1"];
}
else
{
//4rd Point Found
[sortedPoints setObject:[[points objectAtIndex:missingIndexOne] objectForKey:@"point"] forKey:@"1"];
[sortedPoints setObject:[[points objectAtIndex:missingIndexTwo] objectForKey:@"point"] forKey:@"3"];
}
[_adjustRect topLeftCornerToCGPoint:[(NSValue *)[sortedPoints objectForKey:@"0"] CGPointValue]];
[_adjustRect topRightCornerToCGPoint:[(NSValue *)[sortedPoints objectForKey:@"1"] CGPointValue]];
[_adjustRect bottomRightCornerToCGPoint:[(NSValue *)[sortedPoints objectForKey:@"2"] CGPointValue]];
[_adjustRect bottomLeftCornerToCGPoint:[(NSValue *)[sortedPoints objectForKey:@"3"] CGPointValue]];
}
original.release();
}
推荐答案
<这个天真的实现基于 squares.cpp ，可在OpenCV示例目录中找到。以下帖子还讨论了类似的申请：
This naive implementation is based on some of the techniques demonstrated in squares.cpp, available in the OpenCV sample directory. The following posts also discuss similar applications:
- OpenCV C++/Obj-C: Detecting a sheet of paper / Square Detection
- Square detection doesn't find squares
- Find corner of papers
@John，下面的代码已使用您提供的示例图像和我创建的另一个图像进行了测试：
@John, the code below has been tested with the sample image you provided and another one I created:
处理管道以<$ c开头$ c> findSquares（），是OpenCV的 squares.cpp 演示实现的相同功能的简化。此函数将输入图像转换为灰度并应用模糊以改善边缘检测（Canny）：
The processing pipeline starts with findSquares(), a simplification of the same function implemented by OpenCV's squares.cpp demo. This function converts the input image to grayscale and applies a blur to improve the detection of the edges (Canny):
边缘检测很好，但需要进行形态学操作（扩张）才能加入附近的线：
The edge detection is good, but a morphological operation (dilation) is needed to join nearby lines:
之后我们尝试找到轮廓（边缘）并组装出正方形。如果我们试图在输入图像上绘制所有检测到的方块，结果就是：
After that we try to find the contours (edges) and assemble squares out of them. If we tried to draw all the detected squares on the input images, this would be the result:
它看起来不错，但由于检测到的方块太多，它并不是我们想要的。然而，最大的方块实际上是卡片，所以从这里开始它非常简单，我们只是弄清楚哪个方块最大。这正是 findLargestSquare（）所做的。
It looks good, but it's not exactly what we are looking for since there are too many detected squares. However, the largest square is actually the card, so from here on it's pretty simple and we just figure out which of the squares is the largest. That's exactly what findLargestSquare() does.
一旦我们知道最大的正方形，我们只需在广场的角落用于调试目的：
Once we know the largest square, we simply paint red dots at the corners of the square for debugging purposes:
正如您所看到的，对于大多数用途，检测并不完美，但似乎足够好。这不是一个强大的解决方案，我只想分享一种方法来解决问题。我相信还有其他方法可以解决这个问题，对你来说可能更有趣。祝你好运！
As you can see, the detection is not perfect but it seems good enough for most uses. This is not a robust solution and I only wanted to share one approach to solve the problem. I'm sure that there are other ways to deal with this that might be more interesting to you. Good luck!
#include <iostream>
#include <cmath>
#include <vector>
#include <opencv2/highgui/highgui.hpp>
#include <opencv2/imgproc/imgproc.hpp>
#include <opencv2/imgproc/imgproc_c.h>
/* angle: finds a cosine of angle between vectors, from pt0->pt1 and from pt0->pt2
*/
double angle(cv::Point pt1, cv::Point pt2, cv::Point pt0)
{
double dx1 = pt1.x - pt0.x;
double dy1 = pt1.y - pt0.y;
double dx2 = pt2.x - pt0.x;
double dy2 = pt2.y - pt0.y;
return (dx1*dx2 + dy1*dy2)/sqrt((dx1*dx1 + dy1*dy1)*(dx2*dx2 + dy2*dy2) + 1e-10);
}
/* findSquares: returns sequence of squares detected on the image
*/
void findSquares(const cv::Mat& src, std::vector<std::vector<cv::Point> >& squares)
{
cv::Mat src_gray;
cv::cvtColor(src, src_gray, cv::COLOR_BGR2GRAY);
// Blur helps to decrease the amount of detected edges
cv::Mat filtered;
cv::blur(src_gray, filtered, cv::Size(3, 3));
cv::imwrite("out_blur.jpg", filtered);
// Detect edges
cv::Mat edges;
int thresh = 128;
cv::Canny(filtered, edges, thresh, thresh*2, 3);
cv::imwrite("out_edges.jpg", edges);
// Dilate helps to connect nearby line segments
cv::Mat dilated_edges;
cv::dilate(edges, dilated_edges, cv::Mat(), cv::Point(-1, -1), 2, 1, 1); // default 3x3 kernel
cv::imwrite("out_dilated.jpg", dilated_edges);
// Find contours and store them in a list
std::vector<std::vector<cv::Point> > contours;
cv::findContours(dilated_edges, contours, cv::RETR_LIST, cv::CHAIN_APPROX_SIMPLE);
// Test contours and assemble squares out of them
std::vector<cv::Point> approx;
for (size_t i = 0; i < contours.size(); i++)
{
// approximate contour with accuracy proportional to the contour perimeter
cv::approxPolyDP(cv::Mat(contours[i]), approx, cv::arcLength(cv::Mat(contours[i]), true)*0.02, true);
// Note: absolute value of an area is used because
// area may be positive or negative - in accordance with the
// contour orientation
if (approx.size() == 4 && std::fabs(contourArea(cv::Mat(approx))) > 1000 &&
cv::isContourConvex(cv::Mat(approx)))
{
double maxCosine = 0;
for (int j = 2; j < 5; j++)
{
double cosine = std::fabs(angle(approx[j%4], approx[j-2], approx[j-1]));
maxCosine = MAX(maxCosine, cosine);
}
if (maxCosine < 0.3)
squares.push_back(approx);
}
}
}
/* findLargestSquare: find the largest square within a set of squares
*/
void findLargestSquare(const std::vector<std::vector<cv::Point> >& squares,
std::vector<cv::Point>& biggest_square)
{
if (!squares.size())
{
std::cout << "findLargestSquare !!! No squares detect, nothing to do." << std::endl;
return;
}
int max_width = 0;
int max_height = 0;
int max_square_idx = 0;
for (size_t i = 0; i < squares.size(); i++)
{
// Convert a set of 4 unordered Points into a meaningful cv::Rect structure.
cv::Rect rectangle = cv::boundingRect(cv::Mat(squares[i]));
//std::cout << "find_largest_square: #" << i << " rectangle x:" << rectangle.x << " y:" << rectangle.y << " " << rectangle.width << "x" << rectangle.height << endl;
// Store the index position of the biggest square found
if ((rectangle.width >= max_width) && (rectangle.height >= max_height))
{
max_width = rectangle.width;
max_height = rectangle.height;
max_square_idx = i;
}
}
biggest_square = squares[max_square_idx];
}
int main()
{
cv::Mat src = cv::imread("cc.png");
if (src.empty())
{
std::cout << "!!! Failed to open image" << std::endl;
return -1;
}
std::vector<std::vector<cv::Point> > squares;
findSquares(src, squares);
// Draw all detected squares
cv::Mat src_squares = src.clone();
for (size_t i = 0; i < squares.size(); i++)
{
const cv::Point* p = &squares[i][0];
int n = (int)squares[i].size();
cv::polylines(src_squares, &p, &n, 1, true, cv::Scalar(0, 255, 0), 2, CV_AA);
}
cv::imwrite("out_squares.jpg", src_squares);
cv::imshow("Squares", src_squares);
std::vector<cv::Point> largest_square;
findLargestSquare(squares, largest_square);
// Draw circles at the corners
for (size_t i = 0; i < largest_square.size(); i++ )
cv::circle(src, largest_square[i], 4, cv::Scalar(0, 0, 255), cv::FILLED);
cv::imwrite("out_corners.jpg", src);
cv::imshow("Corners", src);
cv::waitKey(0);
return 0;
}
这篇关于检测带圆角的卡片边缘的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
