从轮廓OpenCV中检测卡MinArea Quadrilateral [英] Detect card MinArea Quadrilateral from contour OpenCV
问题描述
另一个关于检测图片中的卡片的问题。
我已经成功地将图片中的卡片隔离开了，我有一个靠近的凸包，从这里我就卡住了。
Yet another about detecting card in a picture. I've managed to pretty much isolate the card in the picture, I have a convex hull that is close and from here I'm stuck.
For上下文/约束，目标：
For the context/constraint, objective:
- 检测图片中的卡片
- 平原背景（见例）
- 前面固定的卡片类型（意思是：我们有宽/高比率）
- 每张图片一个对象（目前在至少）
- Detect a card in a picture
- Plain-ish background (see example)
- Type of card fixed ahead (meaning: we have the width/height ratio)
- One object per picture (for now at least)
我用过的方法：
- Downscale
- 灰度
- 光线模糊
- Canny
- 查找轮廓
- 删除列表中少于120个点的所有轮廓（尝试/错误值）
- 案例1：我有1个轮廓：我卡的完美轮廓：第9步
- 案例2：我有多个轮廓
- 凸壳
- 近似多边形？
- Downscale
- Gray-scale
- Light Blur
- Canny
- Find contours
- Remove all contours in list with less than 120 points (try/error value)
- Case 1: I have 1 contour: perfect contour of my card: step 9
- Case 2: I have multiple contour
- Convex hull
- Approximate polygon ?
步骤1,3和6主要是去除噪音和小文物。
Step 1, 3 and 6 are mainly to remove noise and small artifacts.
所以我几乎停留在第9步。
我试过样本图片：
So I'm pretty much stuck at step 9. I've tried on a sample picture:
在调试图片上：
- 绿色：轮廓
- 红色：凸包
- 紫色/粉红色：用过aboutPolyDp
- 黄色：minAreaRect
- Green: contours
- Red: convex hull
- Purple/Pink-ish: used approxPolyDp
- Yellow: minAreaRect
（结果图像是从minAreaRect中提取的）
(the result image is extracted from the minAreaRect)
所以轮廓是可以接受的，我可以通过调整canny或第一次模糊的参数来做得更好。
但是现在这是可以接受的，现在的问题是，我怎样才能得到形成minarea quadrilateral的4个点。
正如你所看到的，minAreaRect给出了一个不完美的矩形，而且aboutPolyDp丢失了太多的卡片。
So the contour is acceptable, I can probably do a little better by tweaking the parameters from canny or the first blur. But for now this is acceptable, the issue now is, how can I get the the 4 points that will form the "minarea quadrilateral". As you can see, minAreaRect gives a rectangle which is not perfect, and the approxPolyDp is losing too much of the card.
任何线索如何接近这个？
我在使用aboutPolyDp时尝试使用epsilon值（我使用 arcLength * 0.1 ），但没有。
Any clue how I can approach this?
I tried playing with the epsilon value when using approxPolyDp (I used arcLength*0.1), but nothing.
这种方法的另一个问题是在canny期间丢失了一个角落（参见示例）它将不起作用（除非使用minAreaRect时）。但这可能在之前（通过更好的预处理）或之后（因为我们知道宽度/高度比）得到解决。
Another issue with this approach is that is a corner is lost during canny (see example) it'll not work (unless when using minAreaRect). But this can probably be resolved before (with a better pre-processing) or after (since we know the width/height ratio).
这里没有要求代码，只是想法如何处理这个问题，
Not asking for code here, just ideas how to approach this,
谢谢！
编辑：Yves Daoust的解决方案：
Edit: Yves Daoust's solutions:
- 从与谓词匹配的凸包获得8个点：$ b $ b（最大化x，x + y，y，-x + y，-x，-xy，-y，xy）
- 从这个八边形，取4个最长边，得到交叉点
结果：
编辑2：使用Hough变换（而不是8个极值点） ）让我更好找到4个边的所有情况的结果。如果找到超过4行，可能我们有重复，所以使用一些数学来尝试过滤并保留4行。我使用行列式编写了一个草稿（如果平行则接近0）和点线距离公式
Edit 2: Using Hough transform (instead of 8 extreme points) gives me better result for all cases where the 4 sides are found. If more than 4 lines are found, probably we have duplicates, so use some maths to try to filter and keep 4 lines. I coded a draft working by using the determinant (close to 0 if parallel) and the point-line distance formula)
推荐答案
这里是我在输入图像上尝试的管道：
Here is the pipeline I tried on your input image:
- 模糊灰度输入并使用 Canny过滤器检测边缘
- Blur grayscale input and detect edges with Canny filter
- 计算轮廓
- 按长度排序轮廓<仅保持最大
- 生成凸包 strong>此轮廓
- 从凸包中创建蒙版
- 使用
HoughLinesP查找卡片的 4面 - 计算4面的交叉点
- Compute the contours
- Sort the contours by length and only keep the largest one
- Generate the convex hull of this contour
- Create a mask out of the convex hull
- Use
HoughLinesPto find the 4 sides of your cards - Compute the intersections of the 4 sides
- 使用
findHomography查找卡片的仿射转换（在步骤2 处找到4个交叉点） ） - Warp 使用计算单应矩阵的输入图像
- Use
findHomographyto find the affine transformation of your card (with the 4 intersection points found at Step 2) - Warp the input image using the computed homography matrix
结果如下：
And here is the result:
请注意，您必须找到的方法对4个交叉点进行排序，以便总是以相同的顺序排列（否则 findHomography 将无效）。
Note that you will have to find a way to sort the 4 intersection points so that there are always in the same order (otherwise findHomography won't work).
我知道你没有要求代码，但我必须测试我的管道所以这里是...... :)
I know you didn't ask for code, but I had to test my pipeline so here it is... :)
Vec3f calcParams(Point2f p1, Point2f p2) // line's equation Params computation
{
float a, b, c;
if (p2.y - p1.y == 0)
{
a = 0.0f;
b = -1.0f;
}
else if (p2.x - p1.x == 0)
{
a = -1.0f;
b = 0.0f;
}
else
{
a = (p2.y - p1.y) / (p2.x - p1.x);
b = -1.0f;
}
c = (-a * p1.x) - b * p1.y;
return(Vec3f(a, b, c));
}
Point findIntersection(Vec3f params1, Vec3f params2)
{
float x = -1, y = -1;
float det = params1[0] * params2[1] - params2[0] * params1[1];
if (det < 0.5f && det > -0.5f) // lines are approximately parallel
{
return(Point(-1, -1));
}
else
{
x = (params2[1] * -params1[2] - params1[1] * -params2[2]) / det;
y = (params1[0] * -params2[2] - params2[0] * -params1[2]) / det;
}
return(Point(x, y));
}
vector<Point> getQuadrilateral(Mat & grayscale, Mat& output) // returns that 4 intersection points of the card
{
Mat convexHull_mask(grayscale.rows, grayscale.cols, CV_8UC1);
convexHull_mask = Scalar(0);
vector<vector<Point>> contours;
findContours(grayscale, contours, RETR_EXTERNAL, CHAIN_APPROX_NONE);
vector<int> indices(contours.size());
iota(indices.begin(), indices.end(), 0);
sort(indices.begin(), indices.end(), [&contours](int lhs, int rhs) {
return contours[lhs].size() > contours[rhs].size();
});
/// Find the convex hull object
vector<vector<Point> >hull(1);
convexHull(Mat(contours[indices[0]]), hull[0], false);
vector<Vec4i> lines;
drawContours(convexHull_mask, hull, 0, Scalar(255));
imshow("convexHull_mask", convexHull_mask);
HoughLinesP(convexHull_mask, lines, 1, CV_PI / 200, 50, 50, 10);
cout << "lines size:" << lines.size() << endl;
if (lines.size() == 4) // we found the 4 sides
{
vector<Vec3f> params(4);
for (int l = 0; l < 4; l++)
{
params.push_back(calcParams(Point(lines[l][0], lines[l][1]), Point(lines[l][2], lines[l][3])));
}
vector<Point> corners;
for (int i = 0; i < params.size(); i++)
{
for (int j = i; j < params.size(); j++) // j starts at i so we don't have duplicated points
{
Point intersec = findIntersection(params[i], params[j]);
if ((intersec.x > 0) && (intersec.y > 0) && (intersec.x < grayscale.cols) && (intersec.y < grayscale.rows))
{
cout << "corner: " << intersec << endl;
corners.push_back(intersec);
}
}
}
for (int i = 0; i < corners.size(); i++)
{
circle(output, corners[i], 3, Scalar(0, 0, 255));
}
if (corners.size() == 4) // we have the 4 final corners
{
return(corners);
}
}
return(vector<Point>());
}
int main(int argc, char** argv)
{
Mat input = imread("playingcard_input.png");
Mat input_grey;
cvtColor(input, input_grey, CV_BGR2GRAY);
Mat threshold1;
Mat edges;
blur(input_grey, input_grey, Size(3, 3));
Canny(input_grey, edges, 30, 100);
vector<Point> card_corners = getQuadrilateral(edges, input);
Mat warpedCard(400, 300, CV_8UC3);
if (card_corners.size() == 4)
{
Mat homography = findHomography(card_corners, vector<Point>{Point(warpedCard.cols, 0), Point(warpedCard.cols, warpedCard.rows), Point(0,0) , Point(0, warpedCard.rows)});
warpPerspective(input, warpedCard, homography, Size(warpedCard.cols, warpedCard.rows));
}
imshow("warped card", warpedCard);
imshow("edges", edges);
imshow("input", input);
waitKey(0);
return 0;
}
编辑：我已经调整了一点 Canny 和 HoughLinesP 的参数可以更好地检测卡片（程序现在适用于两个输入样本） 。
I've have tweaked a little the parameters of Canny and HoughLinesP functions to have a better detection of the card (program now works on both input samples).
这篇关于从轮廓OpenCV中检测卡MinArea Quadrilateral的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
