将四边形（四角形）拟合到斑点 [英] Fit Quadrilateral (Tetragon) to a blob

问题描述

应用了不同的过滤和分段技术后，最终得到的图像如下：

我可以使用一些轮廓检测功能，该功能返回该对象边缘上的点列表，或者返回一个拟合的多边形（尽管有很多边缘，但多于4个）。
我想要一种使四边形适合该形状的方法，因为我知道它是鞋盒的前表面，应该是四边形。由于透视图的原因，并行性是不守恒的，因此我现在没有任何限制，只需要四个线段围绕此框即可。

到目前为止，我能找到的是仅矩形拟合，并不能真正返回我需要的结果，因为它会迫使拟合的四边形变为矩形。

如果我可以使用摄像机的相对角度到该鞋盒并知道该鞋盒与我的相机的距离，我可以生成一个全息矩阵并扭曲图像，使该鞋盒再次显示为矩形，但是现在我无法访问此类信息，并且想要纯粹基于视觉来进行操作。

有解决这种问题的已知方法吗？

解决方案

我建议执行以下步骤：

1. threshold（）图片



2. 
   
   

   步骤2：扩张

    ＃扩展阈值图像-合并顶部/底部
   内核= np.ones（（3,3），np.uint8）
   扩张= cv2.dilate（阈值，内核，迭代次数= 3）
    cv2.imshow（'阈值扩张'，膨胀）
     

   
   
   

   第3步：找到轮廓

    ＃找到轮廓
   轮廓，层次结构= cv2.findContours（散开的cv2.RETR_EXTERNAL，cv2.CHAIN_APPROX_SIMPLE）
    cv2.drawContours（img，contours，0，（255,255,255），3）
   打印 contours：，len（contours） 
   打印最大轮廓具有，len（contours [0]），点 
     

   
   
   

   请注意，先进行扩张，然后使用简单的外部轮廓即可塑造形状，但是仍然很复杂（污染279点）

   
   
   

   从现在开始，您可以进一步

   
   
   

   可以是

   
   
   

   b：凸包

    ＃凸出壳
   壳= cv2.convexHull（contours [0]）
    cv2.drawContours（img，[hull]，0，（255,255,255），3）
   打印凸包具有，len（hull），点 
     

   
   
   

   更好，但是您仍然需要处理22个点，并且它并不紧密，

   
   
   

   c：简化放大轮廓

    ＃简化轮廓
    
    epsilon = 0.1 * cv2.arcLength（contours [0]，真）
   大约= cv2.approxPolyDP（轮廓[0]，ε，真）
    cv2.drawContours（img，[大约]，0，（255,255,255），3）
   打印简化的轮廓具有，len（大约），点 
     

   
   
   

   这可能是您追求的目标：仅4分。
   如果需要更多积分，则可以使用epsilon值。

   
   
   

   请记住，现在有一个四边形，但是图片变平了：没有有关透视图/ 3d旋转的信息。

   
   
   

   完整的OpenCV Python代码清单（根据需要注释/取消注释，请使用引用来适应c ++ / java / etc。）

    将numpy导入为np 
    import cv2 
    
    img = cv2.imread（'XwzWQ.png '，0）
    
   ＃阈值图像
    ret，thresh = cv2.threshold（img，127,255,0）
    cv2.imshow（'threshold'，thresh）
    
   ＃扩张阈值图像-合并顶部/底部
   内核= np.ones（（3,3），np.uint8）
   扩张= cv2.dilate（阈值，内核，迭代次数= 3）
    cv2.imshow（'阈值扩张'，扩张）
    
   ＃查找轮廓
   ＃cv2.drawContours（img，Silhouettes，0，（255,255,255），3）
   打印 contours：，len（contours）
   打印最大轮廓具有，len（等高线[0]），点 
    
   ＃minAreaRect 
   ＃rect = cv2.minAreaRect（contours [0]）
   ＃box = cv2.cv.BoxPoints（rect） 
   ＃box = np.int0（box）
   ＃cv2.drawContours（img，[box]，0，（255,255,255），3）
    
   ＃凸外壳
    ＃hull = cv2.convexHull（contours [0]）
   ＃cv2.drawContours（img，[hull]，0，（255,255,255），3）
   ＃打印凸出的外壳具有，len（hull ），点 
    
   ＃简化轮廓
    epsilon = 0.1 * cv2.arcLength（contours [0]，True）
   rox = cv2.approxPolyDP（contours [0]， epsilon，True）
    cv2.drawContours（img，[approx]，0，（255,255,255），3）
   打印简化轮廓具有，len（大约），点 
    
    
   ＃显示输出
    cv2.imshow（'image'，img）
    
    cv2.waitKey（0）
    cv2.destroyAllWindows（）
     

   
   

   After applying different filtering and segmentation techniques, I end up with an image like this:

   I have access to some contours detection functions that return a list of points on the edge of that object, or returns a fitted polygon (with many edges though, much more than 4). I want a way to fit a quadrilateral to that shape as I know it is a front face of a shoebox that is supposed to be a quadrilateral. Due to the perspective view, the parallelity is not conserved so I have no constraints for now and just need four line segments encompassing this box.

   What I could find until now was only rectangle fitting, which doesn't really returns the result I need as it forces the fitted quadrilateral to be rectangular.

   If I had access to the relative angle of the camera to that shoebox and knew the distance of the shoebox from my camera, I could generate a Homography matrix and warp the image such that the shoebox appears rectangular again, but for now I have no access to such information and want to do it based purely vision.

   Any known approaches for solving such a problem?

   解决方案

   I recommend the following steps:

   1. threshold() the image
   2. dilate() the image - this will remove the black line splitting the top and bottom section and also darker artifacts on the lower part
   3. findContours() using setting to retrieve only external contours(RETR_EXTERNAL) and simplify the output(CHAIN_APPROX_SIMPLE)
   4. process the contours further

   Step 1:threshold

   # threshold image
   ret,thresh = cv2.threshold(img,127,255,0)
   cv2.imshow('threshold ',thresh)
   

   Step 2:dilate

   # dilate thresholded image - merges top/bottom 
   kernel = np.ones((3,3), np.uint8)
   dilated = cv2.dilate(thresh, kernel, iterations=3)
   cv2.imshow('threshold dilated',dilated)
   

   Step 3: find contours

   # find contours
   contours, hierarchy = cv2.findContours(dilated,cv2.RETR_EXTERNAL,cv2.CHAIN_APPROX_SIMPLE)
   cv2.drawContours(img, contours, 0, (255,255,255), 3)
   print "contours:",len(contours)
   print "largest contour has ",len(contours[0]),"points"
   

   Notice that dilating first, then using simple external contours gets you shape you're after, but it's still pretty complex (containg 279 points)

   From this point onward you can futher process the contour features. There are a few options, available such as:

   a: getting the min. area rectangle

   # minAreaRect
   rect = cv2.minAreaRect(contours[0])
   box = cv2.cv.BoxPoints(rect)
   box = np.int0(box)
   cv2.drawContours(img,[box],0,(255,255,255),3)
   

   Can be useful, but not exactly what you need.

   b: convex hull

   # convexHull
   hull = cv2.convexHull(contours[0])
   cv2.drawContours(img, [hull], 0, (255,255,255), 3)
   print "convex hull has ",len(hull),"points"
   

   Better, but you still have 22 points to deal with and it's not tight as it could be

   c: simplify contours

   # simplify contours
   
   epsilon = 0.1*cv2.arcLength(contours[0],True)
   approx = cv2.approxPolyDP(contours[0],epsilon,True)
   cv2.drawContours(img, [approx], 0, (255,255,255), 3)
   print "simplified contour has",len(approx),"points"
   

   This is probably what you're after: just 4 points. You can play with the epsilon value if you need more points.

   Bare in mind, now you have a quad, but the picture is flattened: there's no information on perspective/3d rotation.

   Full OpenCV Python code listing (comment/uncomment as needed, use the reference to adapt to c++/java/etc.):

   import numpy as np
   import cv2
   
   img = cv2.imread('XwzWQ.png',0)
   
   # threshold image
   ret,thresh = cv2.threshold(img,127,255,0)
   cv2.imshow('threshold ',thresh)
   
   # dilate thresholded image - merges top/bottom 
   kernel = np.ones((3,3), np.uint8)
   dilated = cv2.dilate(thresh, kernel, iterations=3)
   cv2.imshow('threshold dilated',dilated)
   
   # find contours
   contours, hierarchy = cv2.findContours(dilated,cv2.RETR_EXTERNAL,cv2.CHAIN_APPROX_SIMPLE)
   # cv2.drawContours(img, contours, 0, (255,255,255), 3)
   print "contours:",len(contours)
   print "largest contour has ",len(contours[0]),"points"
   
   # minAreaRect
   # rect = cv2.minAreaRect(contours[0])
   # box = cv2.cv.BoxPoints(rect)
   # box = np.int0(box)
   # cv2.drawContours(img,[box],0,(255,255,255),3)
   
   # convexHull
   # hull = cv2.convexHull(contours[0])
   # cv2.drawContours(img, [hull], 0, (255,255,255), 3)
   # print "convex hull has ",len(hull),"points"
   
   # simplify contours
   epsilon = 0.1*cv2.arcLength(contours[0],True)
   approx = cv2.approxPolyDP(contours[0],epsilon,True)
   cv2.drawContours(img, [approx], 0, (255,255,255), 3)
   print "simplified contour has",len(approx),"points"
   
   
   # display output 
   cv2.imshow('image',img)
   
   cv2.waitKey(0)
   cv2.destroyAllWindows()
   

   这篇关于将四边形（四角形）拟合到斑点的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！