python opencv中的不规则形状检测和测量 [英] Irregular shape detection and measurement in python opencv
问题描述
我正在尝试在python中使用OpenCV进行一些图像分析，但我认为图像本身会非常棘手，我之前从未做过这样的事情所以我想要发出我的逻辑并且可能在我投入大量时间走错路之前，先获得一些想法/实用代码来实现我想做的事情。
你所看到的是冰晶，我想找到每个冰晶的平均大小。每个界限都有相当明确的定义，所以概念上这是我的方法，如果这是错误的方法，我想听听任何建议或意见：
- RGB中的图像被导入并转换为8位灰色（根据我在ImageJ中的测试，32会更好，但我还没想到如何在OpenCV中做到这一点。）
- 边缘可选择高斯模糊以消除噪音
- Canny边缘检测器拾取线
- 形态变换（侵蚀+膨胀） ）这样做是为了试图进一步缩小边界。
此时似乎我可以做出选择。我可以对图像进行二值化，并测量高于阈值的斑点（即斑点为白色时的最大值像素），或者通过更充分地关闭和填充轮廓来继续边缘检测。虽然查看该教程，但轮廓看起来很复杂，虽然我可以让代码在我的图像上运行，但它并没有正确地检测到晶体（不出所料）。我也不确定在进行二值化之前是否应该进行变换？
假设我可以完成所有工作，我认为合理的措施是最长的最小封闭框或椭圆的轴。
我还没有完全解决所有的阈值，因此有些晶体被遗漏了，但是因为它们是被平均，这不会出现大问题。
脚本存储处理后的图像，所以我也喜欢最终输出图像类似于链接的SO线程中的标记blob图像，但是每个blob都标注了它的尺寸。
这是一个（不完整的）理想化输出的样子比如，每个水晶都被识别，注释和测量（当我走得那么远时，我非常确定我可以解决测量问题。）
图像缩小和以前的代码尝试，因为他们使线程过长并且不再相关。
编辑III：
根据评论，分水岭算法看起来非常接近我所追求的目标。这里的问题是，分配算法所需的标记区域非常困难（
编辑IV
以下是其他一些测试我玩过的图像。它的表现比我预期的更小的水晶要好得多，显然很多可以通过我尚未尝试过的阈值来完成。
这里是1，从左到右依次对应于Alex在以下步骤中输出的图像。
这是第二个有更大晶体的。
您会注意到这些颜色往往更均匀，但更难辨别边缘。我发现有点令人惊讶的是边缘泛滥对于某些图像有点过于热心，我原本认为对于具有非常微小晶体的图像尤其如此，但实际上它看起来更具有效果在较大的那些。从我们的实际显微镜中可能有很大的空间来改善输入图像的质量，但是编程可以从系统中获得的松弛越多，我们的生活就越容易！
正如我在评论中所提到的，分水岭似乎是解决这个问题的好办法。但是当你回答时，定义标记的前景和背景是困难的部分！我的想法是使用形态梯度沿着冰晶获得良好的边缘并从那里开始工作;形态渐变看起来效果很好。
import numpy as np
import cv2
img = cv2.imread（'image.png'）
blur = cv2.GaussianBlur（img，（7,7），2）
h，w = img.shape [：2]
形态渐变
kernel = cv2.getStructuringElement（cv2.MORPH_ELLIPSE，（7,7））
gradient = cv2.morphologyEx（blur，cv2。 MORPH_GRADIENT，kernel）
cv2.imshow（'Morphological gradient'，gradient）
cv2.waitKey（）
从这里开始，我使用一些阈值来对梯度进行二值化。这可能是一种更简洁的方法......但这比我尝试的其他一些想法更好。
二进制渐变
lowerb = np.array（[0,0,0]）
upperb = np.array（[15,15,15]）
binary = cv2.inRange（gradient，lowerb，upperb）
cv2.imshow（'Binarized gradient'，binary）
cv2.waitKey（）
现在我们遇到了一些问题。它需要一些清理，因为它很乱，而且，图像边缘的冰晶出现了 - 但我们不知道这些晶体实际上在哪里结束所以我们实际上应该忽略那些。为了从掩码中删除它们，我遍历边缘上的像素并使用 floodFill（）将它们从二进制图像中删除。不要在行和列的顺序上混淆; if 语句指定图像矩阵的行和列，而 floodFill（）的输入需要积分（即 x，y 表格，与行，col 相反）。
从边缘填充以去除边缘晶体
范围内的行（h ）：
如果二进制[row，0] == 255：
cv2.floodFill（二进制，无，（0，行），0）
如果二进制[row，w-1] == 255：
cv2.floodFill（binary，None，（w-1，row），0）
for col in range（w）：
if binary [0 ，col] == 255：
cv2.floodFill（binary，None，（col，0），0）
如果二进制[h-1，col] == 255：
cv2。 floodFill（二进制，无，（col，h-1），0）
cv2.imshow（'填充二进制渐变'，二进制）
cv2.waitKey（）
所以这个图像被标记为前景，因为它具有我们想要分割的对象的确定前景。现在我们需要创建一个确定的对象背景。现在，我是以天真的方式做到这一点，这只是为了使你的前景成长，所以你的对象可能都是在那个前景中定义的。但是，您可以使用原始蒙版或甚至渐变以不同的方式来获得更好的定义。不过，这样做还可以，但效果不是很好。
为标签创建背景和未知掩码 b
$ b kernel = cv2.getStructuringElement（cv2.MORPH_ELLIPSE，（17,17））
background = cv2.dilate（foreground，kernel，iterations = 3）
unknown = cv2.subtract （背景，前景）
cv2.imshow（'背景'，背景）
cv2.waitKey（）
因此所有的黑色都是流域的确定背景。我还创建了未知矩阵，它是前景和背景之间的区域，因此我们可以预先标记传递给分水岭的标记，嘿，这些像素肯定在前景，其他像素肯定是背景，我我不确定这些之间。现在剩下要做的就是分水岭！首先，使用连接的组件标记前景图像，识别未知和背景部分，并将其传递到：
Watershed
markers = cv2.connectedComponents（foreground）[1]
markers + = 1＃在所有标签上添加一个，使背景为1，而不是0
个标记[unknown == 255] = 0＃标记未知区域，零
markers = cv2.watershed（img，markers）
您会注意到我在 img 上运行 watershed（）。您可以尝试在图像的模糊版本上运行它（可能是中间模糊 - 我尝试了这个并且为晶体获得了更平滑的边界）或其他预处理版本的图像，这些版本定义了更好的边界或某些东西。
将标记可视化需要一些工作，因为它们都是 uint8 图像中的小数字。所以我所做的就是在0到179之间为它们分配一些色调并在HSV图像中设置，然后转换为BGR以显示标记：
为标记分配0到179之间的色调
hue_markers = np.uint8（179 * np.float32（markers）/ np.max（markers））
blank_channel = 255 * np.ones（（h，w），dtype = np.uint8）
marker_img = cv2.merge（[hue_markers，blank_channel，blank_channel]）
marker_img = cv2.cvtColor（ marker_img，cv2.COLOR_HSV2BGR）
cv2.imshow（'Colored markers'，marker_img）
cv2.waitKey（）
最后，将标记覆盖到原始图像上以检查它们的外观。 / p>
用分水岭标记标记原始图像
labeled_img = img.copy （）
labeled_img [markers> 1] = marker_img [markers> 1]＃1是高建群ound color
labeled_img = cv2.addWeighted（img，0.5，labeled_img，0.5,0）
cv2.imshow（'watershed_result.png'，labeled_img）
cv2.waitKey（）
嗯，这就是整个管道。您应该能够连续复制/粘贴每个部分，并且您应该能够获得相同的结果。该管道中最薄弱的部分是对梯度进行二值化并定义流域的确定背景。距离变换可能在以某种方式二值化梯度时很有用，但我还没有到达那里。无论哪种方式......这是一个很酷的问题，我很想看到你对这个管道所做的任何改变，或者它对其他冰晶图像的影响。
I'm attempting to do some image analysis using OpenCV in python, but I think the images themselves are going to be quite tricky, and I've never done anything like this before so I want to sound out my logic and maybe get some ideas/practical code to achieve what I want to do, before I invest a lot of time going down the wrong path.
This thread comes pretty close to what I want to achieve, and in my opinion, uses an image that should be even harder to analyse than mine. I'd be interested in the size of those coloured blobs though, rather than their distance from the top left. I've also been following this code, though I'm not especially interested in a reference object (the dimensions in pixels alone would be enough for now and can be converted afterwards).
Here's the input image:
What you're looking at are ice crystals, and I want to find the average size of each. The boundaries of each are reasonably well defined, so conceptually this is my approach, and would like to hear any suggestions or comments if this is the wrong way to go:
- Image in RGB is imported and converted to 8bit gray (32 would be better based on my testing in ImageJ, but I haven't figured out how to do that in OpenCV yet).
- The edges are optionally Gaussian blurred to remove noise
- A Canny edge detector picks up the lines
- Morphological transforms (erosion + dilation) are done to attempt to close the boundaries a bit further.
At this point it seems like I have a choice to make. I could either binarise the image, and measure blobs above a threshold (i.e. max value pixels if the blobs are white), or continue with the edge detection by closing and filling contours more fully. Contours seems complicated though looking at that tutorial, and though I can get the code to run on my images, it doesn't detect the crystals properly (unsurprisingly). I'm also not sure if I should morph transform before binarizing too?
Assuming I can get all that to work, I'm thinking a reasonable measure would be the longest axis of the minimum enclosing box or ellipse.
I haven't quite ironed out all the thresholds yet, and consequently some of the crystals are missed, but since they're being averaged, this isn't presenting a massive problem at the moment.
The script stores the processed images as it goes along, so I'd also like the final output image similar to the 'labelled blobs' image in the linked SO thread, but with each blob annotated with its dimensions maybe.
Here's what an (incomplete) idealised output would look like, each crystal is identified, annotated and measured (pretty sure I can tackle the measurement when I get that far).
Abridged the images and previous code attempts as they are making the thread overly long and are no longer that relevant.
Edit III:
As per the comments, the watershed algorithm looks to be very close to achieving what I'm after. The problem here though is that it's very difficult to assign the marker regions that the algorithm requires (http://docs.opencv.org/3.2.0/d3/db4/tutorial_py_watershed.html).
I don't think this is something that can be solved with thresholds through the binarization process, as the apparent colour of the grains varies by much more than the toy example in that thread.
Edit IV
Here are a couple of the other test images I've played with. It fares much better than I expected with the smaller crystals, and theres obviously a lot of finessing that could be done with the thresholds that I havent tried yet.
Here's 1, top left to bottom right correspond to the images output in Alex's steps below.
And here's a second one with bigger crystals.
You'll notice these tend to be more homogeneous in colour, but with harder to discern edges. Something I found a little suprising is that the edge floodfilling is a little overzealous with some of the images, I would have thought this would be particularly the case for the image with the very tiny crystals, but actually it appears to have more of an effect on the larger ones. There is probably a lot of room to improve the quality of the input images from our actual microscopy, but the more 'slack' the programming can take from the system, the easier our lives will be!
As I mentioned in the comments, watershed looks to be an ok approach for this problem. But as you replied, defining the foreground and the background for the markers is the hard part! My idea was to use the morphological gradient to get good edges along the ice crystals and work from there; the morphological gradient seems to work great.
import numpy as np
import cv2
img = cv2.imread('image.png')
blur = cv2.GaussianBlur(img, (7, 7), 2)
h, w = img.shape[:2]
"""Morphological gradient"""
kernel = cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (7, 7))
gradient = cv2.morphologyEx(blur, cv2.MORPH_GRADIENT, kernel)
cv2.imshow('Morphological gradient', gradient)
cv2.waitKey()
From here, I binarized the gradient using some thresholding. There's probably a cleaner way to do this...but this happens to work better than the dozen other ideas I tried.
"""Binarize gradient"""
lowerb = np.array([0, 0, 0])
upperb = np.array([15, 15, 15])
binary = cv2.inRange(gradient, lowerb, upperb)
cv2.imshow('Binarized gradient', binary)
cv2.waitKey()
Now we have a couple issues with this. It needs some cleaning up as it's messy, and further, the ice crystals that are on the edge of the image are showing up---but we don't know where those crystals actually end so we should actually ignore those. To remove those from the mask, I looped through the pixels on the edge and used floodFill() to remove them from the binary image. Don't get confused here on the orders of rows and columns; the if statements are specifying rows and columns of the image matrix, while the input to floodFill() expects points (i.e. x, y form, which is opposite from row, col).
"""Flood fill from the edges to remove edge crystals"""
for row in range(h):
if binary[row, 0] == 255:
cv2.floodFill(binary, None, (0, row), 0)
if binary[row, w-1] == 255:
cv2.floodFill(binary, None, (w-1, row), 0)
for col in range(w):
if binary[0, col] == 255:
cv2.floodFill(binary, None, (col, 0), 0)
if binary[h-1, col] == 255:
cv2.floodFill(binary, None, (col, h-1), 0)
cv2.imshow('Filled binary gradient', binary)
cv2.waitKey()
Great! Now just to clean this up with some opening and closing...
"""Cleaning up mask"""
foreground = cv2.morphologyEx(binary, cv2.MORPH_OPEN, kernel)
foreground = cv2.morphologyEx(foreground, cv2.MORPH_CLOSE, kernel)
cv2.imshow('Cleanup up crystal foreground mask', foreground)
cv2.waitKey()
So this image was labeled as "foreground" because it has the sure foreground of the objects we want to segment. Now we need to create a sure background of the objects. Now, I did this in the naïve way, which just is to grow your foreground a bunch, so that your objects are probably all defined within that foreground. However, you could probably use the original mask or even the gradient in a different way to get a better definition. Still, this works OK, but is not very robust.
"""Creating background and unknown mask for labeling"""
kernel = cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (17, 17))
background = cv2.dilate(foreground, kernel, iterations=3)
unknown = cv2.subtract(background, foreground)
cv2.imshow('Background', background)
cv2.waitKey()
So all the black there is "sure background" for the watershed. Also I created the unknown matrix, which is the area between foreground and background, so that we can pre-label the markers that get passed to watershed as "hey, these pixels are definitely in the foreground, these others are definitely background, and I'm not sure about these ones between." Now all that's left to do is run the watershed! First, you label the foreground image with connected components, identify the unknown and background portions, and pass them in:
"""Watershed"""
markers = cv2.connectedComponents(foreground)[1]
markers += 1 # Add one to all labels so that background is 1, not 0
markers[unknown==255] = 0 # mark the region of unknown with zero
markers = cv2.watershed(img, markers)
You'll notice that I ran watershed() on img. You might experiment running it on a blurred version of the image (maybe median blurring---I tried this and got a little smoother of boundaries for the crystals) or other preprocessed versions of the images which define better boundaries or something.
It takes a little work to visualize the markers as they're all small numbers in a uint8 image. So what I did was assign them some hue in 0 to 179 and set inside a HSV image, then convert to BGR to display the markers:
"""Assign the markers a hue between 0 and 179"""
hue_markers = np.uint8(179*np.float32(markers)/np.max(markers))
blank_channel = 255*np.ones((h, w), dtype=np.uint8)
marker_img = cv2.merge([hue_markers, blank_channel, blank_channel])
marker_img = cv2.cvtColor(marker_img, cv2.COLOR_HSV2BGR)
cv2.imshow('Colored markers', marker_img)
cv2.waitKey()
And finally, overlay the markers onto the original image to check how they look.
"""Label the original image with the watershed markers"""
labeled_img = img.copy()
labeled_img[markers>1] = marker_img[markers>1] # 1 is background color
labeled_img = cv2.addWeighted(img, 0.5, labeled_img, 0.5, 0)
cv2.imshow('watershed_result.png', labeled_img)
cv2.waitKey()
Well, that's the pipeline in it's entirety. You should be able to copy/paste each section in a row and you should be able to get the same results. The weakest parts of this pipeline is binarizing the gradient and defining the sure background for watershed. The distance transform might be useful in binarizing the gradient somehow, but I haven't gotten there yet. Either way...this was a cool problem, I would be interested to see any changes you make to this pipeline or how it fares on other ice-crystal images.
这篇关于python opencv中的不规则形状检测和测量的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
