查找彼此靠近的对象边界 [英] Finding object boundaries which are close to each other
问题描述
我正在研究计算机视觉问题，其中的第一步是找到物体彼此靠近的位置.例如，在下面的图片中，我很有趣地找到用灰色标记的区域.
输入:
输出:
我目前的方法是先反转图像，然后通过侵蚀进行形态梯度跟随，然后去除一些不有趣的轮廓.脚本如下:
img = cv2.imread('mask.jpg'，0)img =(255-img)内核= np.ones(((11,11)，np.uint8)渐变= cv2.morphologyEx(img，cv2.MORPH_GRADIENT，内核)内核= np.ones((5,5)，np.uint8)img_erosion = cv2.erode(梯度，内核，迭代次数= 3)img_erosion [img_erosion>200] = 255img_erosion [img_erosion< = 200] = 0def get_contours(掩码):等高线，层次结构= cv2.findContours(mask，cv2.RETR_TREE，cv2.cv2.CHAIN_APPROX_NONE)返回轮廓cnts = get_contours(img_erosion)img_new = np.zeros_like(img_erosion)img_h，img_w = img_erosion.shape对于我在cnts:如果cv2.contourArea(i)>30:打印(cv2.boundingRect(i)，cv2.contourArea(i))x，y，h，w = cv2.boundingRect(i)如果h/w>5或w/h>5或cv2.contourArea(i)>100:##应该拉长if(x-10> 0)和(y-10> 0):##检查是否靠近上边缘或左边缘if(img_w-x> 10)和(img_h-y> 10):##检查是否在底部或右侧边缘附近cv2.drawContours(img_new，[i]，-1，(255,255,255)，2)内核= np.ones((3,3)，np.uint8)img_new = cv2.dilate(img_new，内核，迭代次数= 2)plt.figure(figsize =(6,6))plt.imshow(img_new) 结果是:
但是，使用这种方法，我需要调整许多参数，并且在方向不同或边缘稍微偏远或者"L"形边缘等情况下，很多情况下都失败了.
我是图像处理的新手，还有其他方法可以帮助我有效地解决此任务吗?
附加更多图像
(大多数为矩形多边形，但是大小和相对位置有很多变化)
最好的 方法是通过
对图像进行阈值处理将告诉您边缘之间相隔很小距离的位置.例如，所有像素的值均小于<比方说，在两个边缘之间的40个像素之间的间隔不足40个像素.
因此，很明显，这很接近您想要的答案.这里会有一些额外的噪音，例如您还将获得形状边缘上的方形脊之间的值...您必须将其过滤掉或平滑掉(轮廓逼近是一种简单的方法，例如，将它们清理为预处理步骤.
但是，尽管我确实编写了SWT原型，但这并不是一个很好的实现，而且我还没有真正测试过它(实际上已经忘记了几个月……也许一年).因此，我现在不打算将其发布.但是，我确实有另一个想法，该想法更简单一些，不需要阅读研究论文.
您的输入图像中有多个斑点.想象一下，如果您每个人都有自己的形象，并且您将每个Blob扩大了您愿意在它们之间放置多大的距离.如果将每个斑点增加10个像素，并且它们重叠，则它们之间的距离应在20个像素之内.但是，这并不能为我们提供完整的重叠区域，而只是两个 expanded 斑点重叠的一部分.一种不同但相似的测量方法是，如果斑点增长了10个像素，并且在原始斑点被扩展之前重叠并且进一步重叠，则两个斑点之间的距离在10像素之内.我们将使用第二个定义来查找附近的斑点.
def find_connection_paths(binimg，距离):h，w = binimg.shape [:2]重叠= np.zeros((h，w)，dtype = np.int32)overlay_mask = np.zeros((h，w)，dtype = np.uint8)内核= cv2.getStructuringElement(cv2.MORPH_ELLIPSE，(距离，距离))#将斑点增加`distance`并求和以得到重叠nlabels，标记为cv2.connectedComponents(binimg，Connectivity = 8)对于范围(1，nlabels)中的标签:掩码= 255 * np.uint8(已标记==标签)重叠+ = cv2.dilate(掩码，内核，迭代次数= 1)//255重叠= np.uint8(重叠> 1)#对于每个重叠，重叠是否会接触到原始斑点?noverlaps，overlap_components = cv2.connectedComponents(重叠，连通性= 8)对于范围(1，noverlaps)中的标签:mask = 255 * np.uint8(overlap_components ==标签)如果np.any(cv2.bitwise_and(binimg，mask)):overlay_mask = cv2.bitwise_or(overlap_mask，mask)返回overlay_mask
现在输出不是完美的-当我扩展Blob时，我用一个圆(膨胀核)向外扩展了它们，因此连接区域并不是十分清晰.但是，这是确保它可以在任何方向上工作的最佳方法.您可能会过滤掉它/将其剪切掉.一种简单的方法是获得每个连接件(以蓝色显示)，并反复将其向下腐蚀一个像素，直到它不与原始斑点重叠.实际上，让我们添加一下:
def find_connection_paths(binimg，距离):h，w = binimg.shape [:2]重叠= np.zeros((h，w)，dtype = np.int32)overlay_mask = np.zeros((h，w)，dtype = np.uint8)overlay_min_mask = np.zeros((h，w)，dtype = np.uint8)kernel_dilate = cv2.getStructuringElement(cv2.MORPH_ELLIPSE，(距离，距离))#将斑点增加`distance`并求和以得到重叠nlabels，标记为= cv2.connectedComponents(binimg)对于范围(1，nlabels)中的标签:掩码= 255 * np.uint8(已标记==标签)重叠+ = cv2.dilate(mask，kernel_dilate，迭代次数= 1)//255重叠= np.uint8(重叠> 1)#对于每个重叠，重叠是否会接触到原始斑点?noverlaps，overlap_components = cv2.connectedComponents(重叠)对于范围(1，noverlaps)中的标签:mask = 255 * np.uint8(overlap_components ==标签)如果np.any(cv2.bitwise_and(binimg，mask)):overlay_mask = cv2.bitwise_or(overlap_mask，mask)#对于每个重叠，缩小直到不接触原始斑点kernel_erode = cv2.getStructuringElement(cv2.MORPH_RECT，(3，3))noverlaps，overlap_components = cv2.connectedComponents(overlap_mask)对于范围(1，noverlaps)中的标签:mask = 255 * np.uint8(overlap_components ==标签)而np.any(cv2.bitwise_and(binimg，mask)):mask = cv2.erode(mask，kernel_erode，迭代次数= 1)overlay_min_mask = cv2.bitwise_or(overlap_min_mask，遮罩)返回overlay_min_mask
当然，如果您仍然希望它们更大或更小，则可以对它们进行任何操作，但这看起来非常接近您要求的输出，因此我将其保留在那里.另外，如果您想知道，我不知道右上角的斑点在哪里.我可以稍后再通过另一张.请注意，最后两个步骤可以合并；检查是否有重叠(如果有的话)，冷却-将其缩小并将其存储在面罩中.
I am working on a computer vision problem, in which one step in the problem is find the locations where the objects are close to each other. Example, in the image below I am interesting in finding the regions marked in gray.
Input :
Output :
My current approach is to first invert the image, followed by morphological gradient follower by erosion, then removing some non-interesting contours. Script is as follows:
img = cv2.imread('mask.jpg', 0)
img = (255 - img)
kernel = np.ones((11,11), np.uint8)
gradient = cv2.morphologyEx(img, cv2.MORPH_GRADIENT, kernel)
kernel = np.ones((5,5), np.uint8)
img_erosion = cv2.erode(gradient, kernel, iterations=3)
img_erosion[img_erosion > 200] = 255
img_erosion[img_erosion <= 200] = 0
def get_contours(mask):
contours, hierarchy = cv2.findContours(mask,cv2.RETR_TREE,cv2.cv2.CHAIN_APPROX_NONE)
return contours
cnts = get_contours(img_erosion)
img_new = np.zeros_like(img_erosion)
img_h, img_w = img_erosion.shape
for i in cnts:
if cv2.contourArea(i) > 30:
print(cv2.boundingRect(i), cv2.contourArea(i))
x, y, h, w = cv2.boundingRect(i)
if h/w > 5 or w/h > 5 or cv2.contourArea(i) > 100: ## Should be elongated
if (x - 10 > 0) and (y - 10 > 0): ## Check if near top or left edge
if (img_w - x > 10) and (img_h - y > 10): ## Check if near bottom or right edge
cv2.drawContours(img_new, [i], -1, (255,255,255), 2)
kernel = np.ones((3,3), np.uint8)
img_new = cv2.dilate(img_new, kernel, iterations=2)
plt.figure(figsize=(6,6))
plt.imshow(img_new)
Result is:
But, using this approach, I am required to adjust many parameters, and it's failing in many cases when the orientation is different or edges are slightly far, or if "L" shaped edges etc.
I am new to image processing, is there any other method that can help me to solve this task efficiently?
Edit : Attaching some more images
(Mostly rectangular polygons, but lot of variation in size and relative positions)
The best way to do this probably is via the Stroke Width Transform. This isn't in OpenCV, though it is in a few other libraries and you can find some implementations floating around the internet. The stroke width transform finds the minimum width between the nearest edges for each pixel in the image. See the following figure from the paper:
Thresholding this image then tells you where there are edges separated by some small distance. E.g., all the pixels with values < 40, say, are between two edges that are separated by less than 40 pixels.
So, as is probably clear, this is pretty close to the answer that you want. There would be some additional noise here, like you'd also get values that are between the square ridges on the edge of your shapes...which you'd have to filter out or smooth out (contour approximation would be a simple way to clean them up as a preprocessing step, for example).
However, while I do have a prototype SWT programmed, it's not a very good implementation, and I haven't really tested it (and actually forgot about it for a few months.......maybe a year) so I'm not going to put it out right now. But, I do have another idea that is a little simpler and doesn't necessitate reading a research paper.
You have multiple blobs in your input image. Imagine if you had each one separately in its own image, and you grew each blob by however much distance you're willing to put between them. If you grew each blob by say 10 pixels, and they overlap, then they'd be within 20 pixels of each other. However this doesn't give us the full overlap region, just a portion of where the two expanded blobs overlapped. A different, but similar way to measure this is if the blobs grew by 10 pixels, and overlapped, and furthermore overlapped the original blobs before they were expanded, then the two blobs are within 10 pixels of each other. We're going to use this second definition to find nearby blobs.
def find_connection_paths(binimg, distance):
h, w = binimg.shape[:2]
overlap = np.zeros((h, w), dtype=np.int32)
overlap_mask = np.zeros((h, w), dtype=np.uint8)
kernel = cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (distance, distance))
# grows the blobs by `distance` and sums to get overlaps
nlabels, labeled = cv2.connectedComponents(binimg, connectivity=8)
for label in range(1, nlabels):
mask = 255 * np.uint8(labeled == label)
overlap += cv2.dilate(mask, kernel, iterations=1) // 255
overlap = np.uint8(overlap > 1)
# for each overlap, does the overlap touch the original blob?
noverlaps, overlap_components = cv2.connectedComponents(overlap, connectivity=8)
for label in range(1, noverlaps):
mask = 255 * np.uint8(overlap_components == label)
if np.any(cv2.bitwise_and(binimg, mask)):
overlap_mask = cv2.bitwise_or(overlap_mask, mask)
return overlap_mask
Now the output isn't perfect---when I expanded the blobs, I expanded them outwardly with a circle (the dilation kernel), so the connection areas aren't exactly super clear. However, this was the best way to ensure it'll work on things of any orientation. You could potentially filter this out/clip it down. An easy way to do this would be to get each connecting piece (shown in blue), and repeatedly erode it down a pixel until it doesn't overlap the original blob. Actually OK let's add that:
def find_connection_paths(binimg, distance):
h, w = binimg.shape[:2]
overlap = np.zeros((h, w), dtype=np.int32)
overlap_mask = np.zeros((h, w), dtype=np.uint8)
overlap_min_mask = np.zeros((h, w), dtype=np.uint8)
kernel_dilate = cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (distance, distance))
# grows the blobs by `distance` and sums to get overlaps
nlabels, labeled = cv2.connectedComponents(binimg)
for label in range(1, nlabels):
mask = 255 * np.uint8(labeled == label)
overlap += cv2.dilate(mask, kernel_dilate, iterations=1) // 255
overlap = np.uint8(overlap > 1)
# for each overlap, does the overlap touch the original blob?
noverlaps, overlap_components = cv2.connectedComponents(overlap)
for label in range(1, noverlaps):
mask = 255 * np.uint8(overlap_components == label)
if np.any(cv2.bitwise_and(binimg, mask)):
overlap_mask = cv2.bitwise_or(overlap_mask, mask)
# for each overlap, shrink until it doesn't touch the original blob
kernel_erode = cv2.getStructuringElement(cv2.MORPH_RECT, (3, 3))
noverlaps, overlap_components = cv2.connectedComponents(overlap_mask)
for label in range(1, noverlaps):
mask = 255 * np.uint8(overlap_components == label)
while np.any(cv2.bitwise_and(binimg, mask)):
mask = cv2.erode(mask, kernel_erode, iterations=1)
overlap_min_mask = cv2.bitwise_or(overlap_min_mask, mask)
return overlap_min_mask
Of course, if you still wanted them to be a little bigger or smaller you could do whatever you like with them, but this looks pretty close to your requested output so I'll leave it there. Also, if you're wondering, I have no idea where the blob on the top right went. I can take another pass at this last piece later. Note that the last two steps could be combined; check if there's overlap, if it is, cool---shrink it down and store it in the mask.
这篇关于查找彼此靠近的对象边界的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
