我如何使用opencv python数数在乐高积木上的孔? [英] How can I count holes on Lego bricks using opencv python?

问题描述

我正在处理我的python项目，在这里我需要计算每个乐高积木装配中有多少个孔.我需要从哪个输入的.json文件中获取有关我需要计算哪个程序集的信息:

 "img_001":[{"red":"0"，"blue":"2"，"white":"1"，"grey":"1"，黄色":"1"}，{"red":"0"，"blue":"1"，"white":"0"，"grey":"1"，黄色":"0" 

所以我需要识别我必须按颜色计数的组件.然后，我必须在特定的砖块装配中添加孔的数量.

这是我使用的图像示例:

我开始将图像更改为hsv颜色空间，并使用轨迹栏找到了每种颜色的遮罩.通过使用 cv2.inRange ，我得到了例如红色的蒙版:如您所见，反射光没有帮助.在这一点上，我不知道该如何前进.我觉得我应该使用 cv2.findContour 来获取每个程序集的轮廓.我当时在考虑直方图均衡化.为了检测圆圈，我想使用 cv2.HoughCircles 或 cv2.SimpleBloopDetector .但是我不知道如何检查每个区域有多少个砖块.输出只是特定装配体中的多个孔.你能给我一些想法吗?哪些OpenCv功能可能在这里适用?您将如何解决这种图像处理问题?感谢您的回答.

解决方案

这是颜色分割的简单但非常有趣的练习.该主题已在各个地方广泛涉及，并围绕到〜42 颜色.很难在下面的图像上可视化结果，但是如果放大，则显示的颜色会比原始图像少:

• 将预处理后的图像转换为HSV颜色空间，以实现更好的颜色分割.

• 由于此方法仅着重于黄砖的分割，因此该算法定义了黄色的低值和高值(在HSV中)以使用此范围对图像进行阈值处理:该范围之外的任何颜色均变为黑色像素.图像编辑器可以帮助您放大原始图像并检查像素的精确HSV值.这是分割的结果:

• 然后处理分割的图像，我们丢弃小的斑点以仅保留最大的斑点(即砖块).通过这种过滤机制，可以计算出有多少黄砖.这是一个不错的技巧:如果您使用 cv2.fillPoly()绘制砖块的轮廓并将其填充为白色，则可以绘制整个砖块，而在单独的图像中没有任何孔创建一个面具.这将很快派上用场！这是黄色面罩的样子:

• 在这一阶段，我们已经确定了图像中所有黄色砖的位置.剩下要做的就是识别每个砖块中的孔.这就是遮罩的所在:如果您注意上面的两个图像，则分割后的图像和遮罩之间的差异主要是砖的孔:

• 处理此图像的轮廓可以丢弃所有不符合孔的小斑点，而仅留下砖的孔.我们可以在分割图像或原始图像上绘制孔的位置以显示它们:

总而言之，此代码提供了一个黄色砖块列表以及另一个包含这些砖块中的孔的列表.从这一点上，这取决于您.该代码可以轻松扩展以处理其他颜色的积木.玩得开心:

  import cv2将numpy导入为np#convertToOpenCVHSV():#从HSV范围转换(H:0-360，S:0-100，V:0-100)#符合OpenCV的期望:(H:0-179，S:0-255，V:0-255)def convertToOpenCVHSV(H，S，V):返回np.array([H//2，S * 2.55，V * 2.55]，np.uint8)#1.加载输入图像img = cv2.imread('test_images/legos.jpg')#2.预处理:量化图像以减少颜色数量div = 6img = img//div * div + div//2cv2.imwrite('lego2_quantized.jpg'，img)#3.转换为HSV颜色空间hsv_img = cv2.cvtColor(img，cv2.COLOR_BGR2HSV)#4.使用预定义的黄色值(最小和最大颜色)分割图像low_yellow = convertToOpenCVHSV(40，35，52)high_yellow = convertToOpenCVHSV(56，95，93)yellow_seg_img = cv2.inRange(hsv_img，low_yellow，high_yellow)#cv2.imshow('yellow_seg_img'，yellow_seg_img)cv2.imwrite('lego4_yellow_seg_img.jpg'，yellow_seg_img)#5.识别并计算黄色积木的数量，并仅用黄色物体创建蒙版bricks_list = []min_size = 5等高线，层次结构= cv2.findContours(yellow_seg_img，cv2.RETR_EXTERNAL，cv2.CHAIN_APPROX_NONE)对于contourIdx，枚举(轮廓)中的cnt:#过滤出很小的段x，y，w，h = cv2.boundingRect(cnt)如果(w< min_size)或(h< min_size):继续#print('contourIdx ='，contourIdx，'w ='，w，'h ='，h)bricks_list.append(cnt)#调试:在原始图像中绘制绿色轮廓#cv2.drawContours(img，cnt，-1，(0，255，0)，2)#绿色print('Detected'，len(bricks_list)，'yellow piece.')#迭代砖块列表，并将其绘制(填充)在新图像上用作遮罩yellow_mask_img = np.zeros((img.shape [0]，img.shape [1])，np.uint8)对于Bricks_list中的cnt:cv2.fillPoly(yellow_mask_img，pts = [cnt]，color =(255,255,255))cv2.imshow('yellow_mask_img'，yellow_mask_img)cv2.imwrite('lego5_yellow_mask_img.jpg'，yellow_mask_img)#debug:仅显示找到的原始黄色积木bricks_img = cv2.bitwise_and(img，img，mask = yellow_mask_img)#cv2.imshow('bricks_img'，bricks_img)cv2.imwrite('lego5_bricks_img.jpg'，bricks_img)#6.识别每个乐高积木中的孔diff_img = yellow_mask_img-yellow_seg_imgcv2.imshow('diff_img'，diff_img)cv2.imwrite('lego6_diff_img.jpg'，diff_img)#debug:创建新的BGR映像以进行调试dbg_img = cv2.cvtColor(yellow_mask_img，cv2.COLOR_GRAY2RGB)#dbg_img = bricks_imghole_list = []min_area_size = 10max_area_size = 24等高线，层次结构= cv2.findContours(yellow_seg_img，cv2.RETR_LIST，cv2.CHAIN_APPROX_NONE)对于contourIdx，枚举(轮廓)中的cnt:#按区域过滤出微小的细分面积= cv2.contourArea(contours [contourIdx])如果(area< min_area_size)或(area> max_area_size):#print('contourIdx ='，outlineIdx，'w ='，w，'h ='，h，'area ='，area，'(ignored)')#cv2.drawContours(dbg_img，cnt，-1，(0，0，255)，2)#红色继续#print('contourIdx ='，contourIdx，'w ='，w，'h ='，h，'area ='，区域)hole_list.append(cnt)#调试:在任何BGR图像上绘制蓝色轮廓，以显示砖的孔对于Holes_list中的cnt:cv2.fillPoly(dbg_img，pts = [cnt]，color =(255，128，0))cv2.fillPoly(img，pts = [cnt]，color =(255，128，0))cv2.imwrite('lego6_dbg_img.jpg'，dbg_img)cv2.imwrite('lego6_img.jpg'，img)#7.遍历孔列表并将它们与特定砖块关联# 去做cv2.imshow('img'，img)cv2.imshow('dbg_img'，dbg_img)cv2.waitKey(0) 

I'm working on my python project where I need to count how many holes are in each assembly of Lego brickets. Information about which assembly I need to count I will take from input .json file which looks like this:

"img_001": [
    {
        "red": "0",
        "blue": "2",
        "white": "1",
        "grey": "1",
        "yellow": "1"
    },
    {
        "red": "0",
        "blue": "1",
        "white": "0",
        "grey": "1",
        "yellow": "0"

So I need to recognize which assembly I have to count by colours. Then I have to and number of holes in particular assembly of brickets.

This is example of image that I work with:

I've started with changing my image to hsv colour space and with using trackbar I found a mask for each colour. With using cv2.inRange I get a mask for example for red color: As you can see reflecting light doesn't help. At this point I don't know how could I move forward. I feel I should use cv2.findContour to get contour of each assembly. I was thinking that Histogram Equalization could be useful here. To detecting circles I want to use cv2.HoughCircles or maybe cv2.SimpleBloopDetector. But I have no idea how could I check how many brickets I have in each area. Output is just a number of holes in particular assembly. Could you get me some ideas? Which OpenCv function may have apply here? How would you solve this kind of image-processing problem? Thanks for your answers.

解决方案

This is a simple but very interesting exercise of color segmentation. This topic has been extensively covered everywhere with several examples spread around Stackoverflow. On many scenarios, color segmentation works best in the HSV color space.

On the left image below you can see the segmentation result of the yellow bricks with blue-ish holes, just to show that they were also detected by this approach.

In this answer I provide a high-level overview of the operations required to detect yellow bricks and identify the holes in them. It does not, however, demonstrates how to count the number of holes inside a particular brick to avoid spoiling your homework. That part I left out of the answer on purpose to leave some work for you to do.

Here are the main steps of my approach:

• Preprocess the image to improve segmentation: the technique used here is called color quantization and it reduces the numbers of colors in the image to ~42 colors. It's hard to visualize the result on the image below but if you zoom in, it displays less colors than the original image:

• Convert the preprocessed image to HSV color space to achieve a better segmentation by color.

• As this approach focus only on the segmentation of yellow bricks, the algorithm defines low and high values of yellow (in HSV) to threshold the image using this range: any color outside the range becomes black pixels. An image editor can help you zoom in on the original image and inspect the exact HSV values of the pixels. Here is the result of the segmentation:

• The segmented image is then processed and we discard small blobs to keep only the largest ones (i.e. the bricks). After this filtering mechanism, its possible to count how many yellow bricks there are. Here comes a nifty trick: if you draw the contour of a brick using cv2.fillPoly() and fill it with white, you'll be able to draw the entire brick without any holes in a separate image to create a mask. This will come in handy very soon! Here is what the yellow mask looks like:

• At this stage we already have the location of all the yellow bricks in the image. All that's left to do is to identify the holes in each brick. That's where the mask comes in: if you pay attention to the two images above, the difference between the segmented image and the mask are mainly the holes of the bricks:

• Processing the contours of this image allows to discard all the small blobs that don't qualify as holes, leaving behind just the holes of the bricks. We can draw the location of holes over the segmented image or over the original image to display them:

In summary, this code offers a list of yellow bricks and another list that contains the holes in those bricks. From this point on it's up to you. The code can be easily expanded to process bricks from other colors. Have fun:

import cv2
import numpy as np

# convertToOpenCVHSV():
#   converts from HSV range (H: 0-360, S: 0-100, V: 0-100)
#   to what OpenCV expects: (H: 0-179, S: 0-255, V: 0-255)
def convertToOpenCVHSV(H, S, V):
    return np.array([H // 2, S * 2.55, V * 2.55], np.uint8)


# 1. Load input image
img = cv2.imread('test_images/legos.jpg')

# 2. Preprocess: quantize the image to reduce the number of colors
div = 6
img = img // div * div + div // 2
cv2.imwrite('lego2_quantized.jpg', img)


# 3. Convert to HSV color space
hsv_img = cv2.cvtColor(img, cv2.COLOR_BGR2HSV)


# 4. Segment the image using predefined values of yellow (min and max colors)
low_yellow = convertToOpenCVHSV(40, 35, 52)
high_yellow = convertToOpenCVHSV(56, 95, 93)
yellow_seg_img = cv2.inRange(hsv_img, low_yellow, high_yellow)
#cv2.imshow('yellow_seg_img', yellow_seg_img)
cv2.imwrite('lego4_yellow_seg_img.jpg', yellow_seg_img)

# 5. Identify and count the number of yellow bricks and create a mask with just the yellow objects
bricks_list = []
min_size = 5

contours, hierarchy = cv2.findContours(yellow_seg_img, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_NONE)
for contourIdx, cnt in enumerate(contours):
    # filter out tiny segments
    x, y, w, h = cv2.boundingRect(cnt)
    if (w < min_size) or (h < min_size):
        continue

    #print('contourIdx=', contourIdx, 'w=', w, 'h=', h)

    bricks_list.append(cnt)

    # debug: draw green contour in the original image
    #cv2.drawContours(img, cnt, -1, (0, 255, 0), 2) # green

print('Detected', len(bricks_list), 'yellow pieces.')

# Iterate the list of bricks and draw them (filled) on a new image to be used as a mask
yellow_mask_img = np.zeros((img.shape[0], img.shape[1]), np.uint8)
for cnt in bricks_list:
    cv2.fillPoly(yellow_mask_img, pts=[cnt], color=(255,255,255))

cv2.imshow('yellow_mask_img', yellow_mask_img)
cv2.imwrite('lego5_yellow_mask_img.jpg', yellow_mask_img)

# debug: display only the original yellow bricks found
bricks_img = cv2.bitwise_and(img, img, mask=yellow_mask_img)
#cv2.imshow('bricks_img', bricks_img)
cv2.imwrite('lego5_bricks_img.jpg', bricks_img)

# 6. Identify holes in each Lego brick
diff_img = yellow_mask_img - yellow_seg_img
cv2.imshow('diff_img', diff_img)
cv2.imwrite('lego6_diff_img.jpg', diff_img)

# debug: create new BGR image for debugging purposes
dbg_img = cv2.cvtColor(yellow_mask_img, cv2.COLOR_GRAY2RGB)
#dbg_img = bricks_img

holes_list = []
min_area_size = 10
max_area_size = 24
contours, hierarchy = cv2.findContours(yellow_seg_img, cv2.RETR_LIST, cv2.CHAIN_APPROX_NONE)
for contourIdx, cnt in enumerate(contours):
    # filter out tiny segments by area
    area = cv2.contourArea(contours[contourIdx])

    if (area < min_area_size) or (area > max_area_size):
        #print('contourIdx=', contourIdx, 'w=', w, 'h=', h, 'area=', area, '(ignored)')
        #cv2.drawContours(dbg_img, cnt, -1, (0, 0, 255), 2) # red
        continue

    #print('contourIdx=', contourIdx, 'w=', w, 'h=', h, 'area=', area)
    holes_list.append(cnt)

# debug: draw a blue-ish contour on any BGR image to show the holes of the bricks
for cnt in holes_list:
    cv2.fillPoly(dbg_img, pts=[cnt], color=(255, 128, 0))
    cv2.fillPoly(img, pts=[cnt], color=(255, 128, 0))

cv2.imwrite('lego6_dbg_img.jpg', dbg_img)
cv2.imwrite('lego6_img.jpg', img)

# 7. Iterate though the list of holes and associate them with a particular brick
# TODO

cv2.imshow('img', img)
cv2.imshow('dbg_img', dbg_img)
cv2.waitKey(0)

这篇关于我如何使用opencv python数数在乐高积木上的孔?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！