从轮廓 OpenCV 检测卡片 MinArea Quadrilateral [英] Detect card MinArea Quadrilateral from contour OpenCV
问题描述
另一个关于检测图片中的卡片.我已经成功地隔离了图片中的卡片,我有一个很近的凸包,从这里我被卡住了.
Yet another about detecting card in a picture. I've managed to pretty much isolate the card in the picture, I have a convex hull that is close and from here I'm stuck.
对于上下文/约束,目标:
For the context/constraint, objective:
- 检测图片中的卡片
- 纯色背景(见示例)
- 固定在前面的卡片类型(意思是:我们有宽高比)
- 每张图片一个对象(至少目前是这样)
我使用的方法:
- 缩小规模
- 灰度
- 轻度模糊
- 精明
- 寻找轮廓
- 删除列表中少于 120 个点的所有轮廓(尝试/错误值)
- 案例 1:我有 1 个轮廓:我的卡片的完美轮廓:第 9 步
- 案例 2:我有多个轮廓
- 凸包
- 近似多边形?
步骤 1、3 和 6 主要是去除噪声和小伪影.
Step 1, 3 and 6 are mainly to remove noise and small artifacts.
所以我几乎卡在了第 9 步.我试过示例图片:
So I'm pretty much stuck at step 9. I've tried on a sample picture:
关于调试图片:
- 绿色:轮廓
- 红色:凸包
- 紫色/粉红色:使用了 approxPolyDp
- 黄色:minAreaRect
(结果图片是从minAreaRect中提取的)
(the result image is extracted from the minAreaRect)
所以轮廓是可以接受的,我可以通过调整 canny 或第一次模糊的参数来做得更好.但现在这是可以接受的,现在的问题是,我怎样才能得到将形成minarea 四边形"的 4 个点.如您所见,minAreaRect 给出了一个不完美的矩形,并且 approxPolyDp 丢失了太多卡片.
So the contour is acceptable, I can probably do a little better by tweaking the parameters from canny or the first blur. But for now this is acceptable, the issue now is, how can I get the the 4 points that will form the "minarea quadrilateral". As you can see, minAreaRect gives a rectangle which is not perfect, and the approxPolyDp is losing too much of the card.
有什么线索可以解决这个问题吗?我在使用 approxPolyDp 时尝试使用 epsilon 值(我使用了 arcLength*0.1
),但没有.
Any clue how I can approach this?
I tried playing with the epsilon value when using approxPolyDp (I used arcLength*0.1
), but nothing.
这种方法的另一个问题是在 canny 期间丢失了一个角(参见示例),它不起作用(除非使用 minAreaRect 时).但这可能可以在之前(通过更好的预处理)或之后解决(因为我们知道宽/高比).
Another issue with this approach is that is a corner is lost during canny (see example) it'll not work (unless when using minAreaRect). But this can probably be resolved before (with a better pre-processing) or after (since we know the width/height ratio).
这里不要求代码,只是想办法解决这个问题,
Not asking for code here, just ideas how to approach this,
谢谢!
编辑:Yves Daoust 的解决方案:
Edit: Yves Daoust's solutions:
- 从凸包中获取与谓词匹配的 8 个点:(最大化 x, x+y, y, -x+y, -x, -x-y, -y, x-y)
- 从这个八边形,取4条最长的边,得到交点
结果:
编辑 2: 使用 Hough 变换(而不是 8 个极值点)可以为找到 4 个边的所有情况提供更好的结果.如果发现超过 4 行,可能我们有重复,所以使用一些数学来尝试过滤并保留 4 行.我使用行列式(如果平行则接近 0)和点线距离公式编写了一个草图)
Edit 2: Using Hough transform (instead of 8 extreme points) gives me better result for all cases where the 4 sides are found. If more than 4 lines are found, probably we have duplicates, so use some maths to try to filter and keep 4 lines. I coded a draft working by using the determinant (close to 0 if parallel) and the point-line distance formula)
推荐答案
这是我在您的输入图像上尝试的管道:
Here is the pipeline I tried on your input image:
- 模糊灰度输入并使用Canny过滤器 检测边缘
- Blur grayscale input and detect edges with Canny filter
- 计算轮廓
- 按长度对轮廓进行排序,只保留最大的
- 生成此轮廓的凸包
- 用凸包创建一个遮罩
- 使用
HoughLinesP
找出你卡片的4 面 - 计算 4 条边的交点
- Compute the contours
- Sort the contours by length and only keep the largest one
- Generate the convex hull of this contour
- Create a mask out of the convex hull
- Use
HoughLinesP
to find the 4 sides of your cards - Compute the intersections of the 4 sides
- 使用
findHomography
找到卡片的仿射变换(在 步骤 2 中找到 4 个交点) - 变形使用计算单应矩阵的输入图像
- Use
findHomography
to find the affine transformation of your card (with the 4 intersection points found at Step 2) - Warp the input image using the computed homography matrix
结果如下:
请注意,您必须找到一种方法来对 4 个交叉点进行排序,以便始终保持相同的顺序(否则 findHomography
将不起作用).
Note that you will have to find a way to sort the 4 intersection points so that there are always in the same order (otherwise findHomography
won't work).
我知道你没有要求代码,但我必须测试我的管道,所以这里是... :)
I know you didn't ask for code, but I had to test my pipeline so here it is... :)
Vec3f calcParams(Point2f p1, Point2f p2) // line's equation Params computation
{
float a, b, c;
if (p2.y - p1.y == 0)
{
a = 0.0f;
b = -1.0f;
}
else if (p2.x - p1.x == 0)
{
a = -1.0f;
b = 0.0f;
}
else
{
a = (p2.y - p1.y) / (p2.x - p1.x);
b = -1.0f;
}
c = (-a * p1.x) - b * p1.y;
return(Vec3f(a, b, c));
}
Point findIntersection(Vec3f params1, Vec3f params2)
{
float x = -1, y = -1;
float det = params1[0] * params2[1] - params2[0] * params1[1];
if (det < 0.5f && det > -0.5f) // lines are approximately parallel
{
return(Point(-1, -1));
}
else
{
x = (params2[1] * -params1[2] - params1[1] * -params2[2]) / det;
y = (params1[0] * -params2[2] - params2[0] * -params1[2]) / det;
}
return(Point(x, y));
}
vector<Point> getQuadrilateral(Mat & grayscale, Mat& output) // returns that 4 intersection points of the card
{
Mat convexHull_mask(grayscale.rows, grayscale.cols, CV_8UC1);
convexHull_mask = Scalar(0);
vector<vector<Point>> contours;
findContours(grayscale, contours, RETR_EXTERNAL, CHAIN_APPROX_NONE);
vector<int> indices(contours.size());
iota(indices.begin(), indices.end(), 0);
sort(indices.begin(), indices.end(), [&contours](int lhs, int rhs) {
return contours[lhs].size() > contours[rhs].size();
});
/// Find the convex hull object
vector<vector<Point> >hull(1);
convexHull(Mat(contours[indices[0]]), hull[0], false);
vector<Vec4i> lines;
drawContours(convexHull_mask, hull, 0, Scalar(255));
imshow("convexHull_mask", convexHull_mask);
HoughLinesP(convexHull_mask, lines, 1, CV_PI / 200, 50, 50, 10);
cout << "lines size:" << lines.size() << endl;
if (lines.size() == 4) // we found the 4 sides
{
vector<Vec3f> params(4);
for (int l = 0; l < 4; l++)
{
params.push_back(calcParams(Point(lines[l][0], lines[l][1]), Point(lines[l][2], lines[l][3])));
}
vector<Point> corners;
for (int i = 0; i < params.size(); i++)
{
for (int j = i; j < params.size(); j++) // j starts at i so we don't have duplicated points
{
Point intersec = findIntersection(params[i], params[j]);
if ((intersec.x > 0) && (intersec.y > 0) && (intersec.x < grayscale.cols) && (intersec.y < grayscale.rows))
{
cout << "corner: " << intersec << endl;
corners.push_back(intersec);
}
}
}
for (int i = 0; i < corners.size(); i++)
{
circle(output, corners[i], 3, Scalar(0, 0, 255));
}
if (corners.size() == 4) // we have the 4 final corners
{
return(corners);
}
}
return(vector<Point>());
}
int main(int argc, char** argv)
{
Mat input = imread("playingcard_input.png");
Mat input_grey;
cvtColor(input, input_grey, CV_BGR2GRAY);
Mat threshold1;
Mat edges;
blur(input_grey, input_grey, Size(3, 3));
Canny(input_grey, edges, 30, 100);
vector<Point> card_corners = getQuadrilateral(edges, input);
Mat warpedCard(400, 300, CV_8UC3);
if (card_corners.size() == 4)
{
Mat homography = findHomography(card_corners, vector<Point>{Point(warpedCard.cols, 0), Point(warpedCard.cols, warpedCard.rows), Point(0,0) , Point(0, warpedCard.rows)});
warpPerspective(input, warpedCard, homography, Size(warpedCard.cols, warpedCard.rows));
}
imshow("warped card", warpedCard);
imshow("edges", edges);
imshow("input", input);
waitKey(0);
return 0;
}
我已经稍微调整了 Canny
和 HoughLinesP
函数的参数,以便更好地检测卡片(程序现在适用于两个输入样本).
I've have tweaked a little the parameters of Canny
and HoughLinesP
functions to have a better detection of the card (program now works on both input samples).
这篇关于从轮廓 OpenCV 检测卡片 MinArea Quadrilateral的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!