如何在OpenCV Python中加入附近的边界框 [英] How to join nearby bounding boxes in OpenCV Python

问题描述

我正在做一个关于图像处理的大学班项目.这是我的原始图片:

I am doing a college class project on image processing. This is my original image:

我想在单个文本行图像上加入附近/重叠的边框，但是我不知道如何.到目前为止，我的代码看起来像这样(感谢@HansHirse的帮助):

I want to join nearby/overlapping bounding boxes on individual text line images, but I don't know how. My code looks like this so far (thanks to @HansHirse for the help):

import os
import cv2
import numpy as np
from scipy import stats
image = cv2.imread('example.png')

gray = cv2.cvtColor(image,cv2.COLOR_BGR2GRAY)
ret,thresh = cv2.threshold(gray,127,255,cv2.THRESH_BINARY_INV+cv2.THRESH_OTSU)

#dilation
kernel = np.ones((5,5), np.uint8)
img_dilation = cv2.dilate(thresh, kernel, iterations=1)

#find contours
ctrs, hier = cv2.findContours(img_dilation.copy(), cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)

# https://www.pyimagesearch.com/2015/04/20/sorting-contours-using-python-and-opencv/
def sort_contours(cnts, method="left-to-right"):
    # initialize the reverse flag and sort index
    reverse = False
    i = 0

    # handle if we need to sort in reverse
    if method == "right-to-left" or method == "bottom-to-top":
        reverse = True

    # handle if we are sorting against the y-coordinate rather than
    # the x-coordinate of the bounding box
    if method == "top-to-bottom" or method == "bottom-to-top":
        i = 1

    # construct the list of bounding boxes and sort them from top to
    # bottom
    boundingBoxes = [cv2.boundingRect(c) for c in cnts]
    (cnts, boundingBoxes) = zip(*sorted(zip(cnts, boundingBoxes),
                                        key=lambda b: b[1][i], reverse=reverse))

    # return the list of sorted contours and bounding boxes
    return (cnts, boundingBoxes)

sortedctrs,sortedbbs=sort_contours(ctrs)
xyminmax=[]
for cnt in sortedctrs:
    x, y, w, h = cv2.boundingRect(cnt)
    xyminmax.append([x,y,x+w,y+h])

distances=[]
for i in range(len(xyminmax)):
    try:
        first_xmax = xyminmax[i][2]
        second_xmin = xyminmax[i + 1][0]
        distance=abs(second_xmin-first_xmax)
        distances.append(distance)
    except IndexError:
        pass

THRESHOLD=stats.mode(distances, axis=None)[0][0]

new_rects=[]
for i in range(len(xyminmax)):
    try:
        # [xmin,ymin,xmax,ymax]
        first_ymin=xyminmax[i][1]
        first_ymax=xyminmax[i][3]

        second_ymin=xyminmax[i+1][1]
        second_ymax=xyminmax[i+1][3]

        first_xmax = xyminmax[i][2]
        second_xmin = xyminmax[i+1][0]

        firstheight=abs(first_ymax-first_ymin)
        secondheight=abs(second_ymax-second_ymin)

        distance=abs(second_xmin-first_xmax)

        if distance<THRESHOLD:
            new_xmin=xyminmax[i][0]
            new_xmax=xyminmax[i+1][2]
            if first_ymin>second_ymin:
                new_ymin=second_ymin
            else:
                new_ymin = first_ymin

            if firstheight>secondheight:
                new_ymax = first_ymax
            else:
                new_ymax = second_ymax
            new_rects.append([new_xmin,new_ymin,new_xmax,new_ymax])
        else:
            new_rects.append(xyminmax[i])
    except IndexError:
        pass

for rect in new_rects:
    cv2.rectangle(image, (rect[0], rect[1]), (rect[2], rect[3]), (121, 11, 189), 2)
cv2.imwrite("result.png",image) 

产生以下图像:

我想加入诸如此类的非常接近或重叠的边界框

I want to join very close or overlapping bounding boxes such as these

放入单个边界框，这样公式就不会分成单个字符.我尝试使用cv2.groupRectangles，但print结果仅为NULL.

into a single bounding box so the formula doesn't get separated into single characters. I have tried using cv2.groupRectangles but the print results were just NULL.

推荐答案

因此，这是我的解决方案.我将您的(初始)代码部分修改为我的首选命名，等等.此外，我还对所有内容进行了评论.

So, here comes my solution. I partially modified your (initial) code to my preferred naming, etc. Also, I commented all the stuff, I added.

import cv2
import numpy as np

image = cv2.imread('images/example.png')

gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
_, thresh = cv2.threshold(gray, 127, 255, cv2.THRESH_BINARY_INV + cv2.THRESH_OTSU)

kernel = np.ones((5, 5), np.uint8)
img_dilated = cv2.dilate(thresh, kernel, iterations = 1)

cnts, _ = cv2.findContours(img_dilated.copy(), cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)

# Array of initial bounding rects
rects = []

# Bool array indicating which initial bounding rect has
# already been used
rectsUsed = []

# Just initialize bounding rects and set all bools to false
for cnt in cnts:
    rects.append(cv2.boundingRect(cnt))
    rectsUsed.append(False)

# Sort bounding rects by x coordinate
def getXFromRect(item):
    return item[0]

rects.sort(key = getXFromRect)

# Array of accepted rects
acceptedRects = []

# Merge threshold for x coordinate distance
xThr = 5

# Iterate all initial bounding rects
for supIdx, supVal in enumerate(rects):
    if (rectsUsed[supIdx] == False):

        # Initialize current rect
        currxMin = supVal[0]
        currxMax = supVal[0] + supVal[2]
        curryMin = supVal[1]
        curryMax = supVal[1] + supVal[3]

        # This bounding rect is used
        rectsUsed[supIdx] = True

        # Iterate all initial bounding rects
        # starting from the next
        for subIdx, subVal in enumerate(rects[(supIdx+1):], start = (supIdx+1)):

            # Initialize merge candidate
            candxMin = subVal[0]
            candxMax = subVal[0] + subVal[2]
            candyMin = subVal[1]
            candyMax = subVal[1] + subVal[3]

            # Check if x distance between current rect
            # and merge candidate is small enough
            if (candxMin <= currxMax + xThr):

                # Reset coordinates of current rect
                currxMax = candxMax
                curryMin = min(curryMin, candyMin)
                curryMax = max(curryMax, candyMax)

                # Merge candidate (bounding rect) is used
                rectsUsed[subIdx] = True
            else:
                break

        # No more merge candidates possible, accept current rect
        acceptedRects.append([currxMin, curryMin, currxMax - currxMin, curryMax - curryMin])

for rect in acceptedRects:
    img = cv2.rectangle(image, (rect[0], rect[1]), (rect[0] + rect[2], rect[1] + rect[3]), (121, 11, 189), 2)

cv2.imwrite("images/result.png", image)

以您为例

我得到以下输出

现在，您必须找到合适的阈值来满足您的期望.也许，还有更多的工作要做，特别是要获得整个公式，因为距离变化不大.

Now, you have to find a proper threshold to meet your expectations. Maybe, there is even some more work to do, especially to get the whole formula, since the distances don't vary that much.

免责声明:我是Python的新手，尤其是OpenCV(胜利的C ++)的Python API.非常欢迎评论，改进，强调Python的执行！！

Disclaimer: I'm new to Python in general, and specially to the Python API of OpenCV (C++ for the win). Comments, improvements, highlighting Python no-gos are highly welcome!

这篇关于如何在OpenCV Python中加入附近的边界框的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！