减少模板匹配中的遗漏(使用透明模板) [英] Reducing misses on template matching (using transparent template)

问题描述

我正在尝试确定拼图图像上拼图的位置.我有

1. 拼图块的图像(透明png)
2. 在正确位置周围带有白色轮廓的拼图板的图像

首先，我在图像1上提取轮廓，然后用它绘制最终模板.然后，我将最终模板匹配到拼图板上，并保存一些结果图像.

以下是一些我得到的结果的例子

小图片:

  import cv2将numpy导入为np#读取图像full_image = cv2.imread('puzzle_1.jpg')一块= cv2.imread('piece_1.png'，cv2.IMREAD_UNCHANGED)#将片段转换为灰色，将阈值转换为二进制局部图像= cv2.cvtColor(piece，cv2.COLOR_BGR2GRAY)partial_image = cv2.threshold(partial_image，0，255，cv2.THRESH_BINARY)[1]#从二进制图像获取最大轮廓等高线= cv2.findContours(partial_image.copy()，cv2.RETR_EXTERNAL，cv2.CHAIN_APPROX_SIMPLE)轮廓=轮廓[0]如果len(contours)== 2，则轮廓[1]big_contour = max(轮廓，键= cv2.contourArea)#绘制工件轮廓的轮廓作为遮罩mask = np.zeros((55，55，3)，dtype = np.uint8)cv2.drawContours(mask，[big_contour]，0，(255,255,255)，1)嗯，ww = mask.shape [:2]#从BGR(无alpha)片段中提取模板模板= piece [:，:，0:3]相关性= cv2.matchTemplate(完整图像，模板，cv2.TM_CCORR_NORMED，mask = mask)min_val，max_val，min_loc，max_loc = cv2.minMaxLoc(correlation)max_val_ncc ='{:.3f}'.format(max_val)打印(" normalize_cross_correlation:" + max_val_ncc)xx = max_loc [0]yy = max_loc [1]打印('xmatchloc ='，xx，'ymatchloc ='，yy)#在img上以红色绘制模板边界和角交叉点结果= full_image.copy()cv2.rectangle(结果，(xx，yy)，(xx + ww，yy + hh)，(0，0，255)，1)#保存结果cv2.imwrite('puzzle_template.png'，模板)cv2.imwrite('puzzle_mask.png'，遮罩)cv2.imwrite('full_image_matches.jpg'，结果)#显示结果cv2.imshow('full_image'，full_image)cv2.imshow('piece'，piece)cv2.imshow('partial_image'，partial_image)cv2.imshow('遮罩'，遮罩)cv2.imshow('template'，template)cv2.imshow('结果'，结果)cv2.waitKey(0)cv2.destroyAllWindows() 

(bgr)模板:

面具:

最大比赛位置:

I'm trying to determine the position of puzzle piece on a puzzle image. I have

1. an image of a puzzle piece (transparent png)
2. an image of a puzzleboard whith a white outline around the correct position

Firstly i extract the contour on image 1. and use it to draw my final template. Then i match the final template to the puzzleboard and save some result images.

Here are some examples of the results i'm getting https://imgur.com/a/ZYyw7tU

On some of the images, there's clearly a lot of mismatches and for some of the images, increasing the threshold will hide the correct match and some mismatches will remain.

Any tips or thoughts on optimization would be greatly appreciated!

This is my full code:

full_image = cv2.imread('puzzle_1.jpg')    
piece = cv2.imread('piece_1.png', cv2.IMREAD_UNCHANGED)
partial_image = cv2.cvtColor(piece,cv2.COLOR_BGR2GRAY)

contours, hierarchy = cv2.findContours(partial_image.copy(), cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_NONE)
   
template = np.zeros((55, 55, 4), dtype=np.uint8)

cv2.drawContours(template, contours, -1, (255, 255, 255, 255),1)
hh, ww = template.shape[:2]

puzzleP = template[:,:,0:3]
alpha = template[:,:,3]
alpha = cv2.merge([alpha,alpha,alpha])

correlation = cv2.matchTemplate(full_image, puzzleP, cv2.TM_CCORR_NORMED, mask=alpha)

threshhold = 0.98
loc = np.where(correlation >= threshhold)

result = full_image.copy()
for pt in zip(*loc[::-1]):
    cv2.rectangle(result, pt, (pt[0]+ww, pt[1]+hh), (0,0,255), 1)
    print(pt)
    
cv2.imwrite('puzzle_piece.png', puzzleP)
cv2.imwrite('full_image_alpha.png', alpha)
cv2.imwrite('full_image_matches.jpg', result)

EDIT: Here's an example of the two files (piece_1.png and puzzle_1.jpg) (this example has many mismatches) https://imgur.com/a/nGSXcNg

解决方案

This seems to work fine for me on your given image in Python/OpenCV.

• Read the large and small images
• Convert the small image to gray and threshold to binary
• Get the largest contour of the binary image
• Draw that contour as white on black background to use as the mask
• Extract the BGR channels of the transparent small image as the template
• Do the template matching and get the largest match location
• Draw the match box on a copy of the large image
• Save the results

Large image:

Small image:

import cv2
import numpy as np

# read images
full_image = cv2.imread('puzzle_1.jpg')    
piece = cv2.imread('piece_1.png', cv2.IMREAD_UNCHANGED)

# convert piece to gray and threshold to binary
partial_image = cv2.cvtColor(piece,cv2.COLOR_BGR2GRAY)
partial_image = cv2.threshold(partial_image, 0, 255, cv2.THRESH_BINARY)[1]

# get largest contour from binary image
contours = cv2.findContours(partial_image.copy(), cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
contours = contours[0] if len(contours) == 2 else contours[1]
big_contour = max(contours, key=cv2.contourArea)

# draw the contour of the piece outline as the mask
mask = np.zeros((55, 55, 3), dtype=np.uint8)
cv2.drawContours(mask, [big_contour], 0, (255,255,255), 1)
hh, ww = mask.shape[:2]

# extract the template from the BGR (no alpha) piece 
template = piece[:,:,0:3]

correlation = cv2.matchTemplate(full_image, template, cv2.TM_CCORR_NORMED, mask=mask)
min_val, max_val, min_loc, max_loc = cv2.minMaxLoc(correlation)
max_val_ncc = '{:.3f}'.format(max_val)
print("normalize_cross_correlation: " + max_val_ncc)
xx = max_loc[0]
yy = max_loc[1]

print('xmatchloc =',xx,'ymatchloc =',yy)

# draw template bounds and corner intersection in red onto img
result = full_image.copy()
cv2.rectangle(result, (xx, yy), (xx+ww, yy+hh), (0, 0, 255), 1)

# save results
cv2.imwrite('puzzle_template.png', template)
cv2.imwrite('puzzle_mask.png', mask)
cv2.imwrite('full_image_matches.jpg', result)

# show results
cv2.imshow('full_image', full_image)
cv2.imshow('piece', piece)
cv2.imshow('partial_image', partial_image)
cv2.imshow('mask', mask)
cv2.imshow('template', template)
cv2.imshow('result', result)
cv2.waitKey(0)
cv2.destroyAllWindows()

(bgr) Template:

Mask:

Largest Match Location:

这篇关于减少模板匹配中的遗漏(使用透明模板)的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！