将边界框合并为一个 [英] Merge the Bounding boxes near by into one

问题描述

我是python的新手，我使用的是快速入门：使用计算机视觉中的REST API和Python提取打印文本（OCR），以便在Sales Fliers中检测文本。因此，该算法的坐标为Ymin，XMax，Ymin，和Xmax并为每行文本绘制一个边界框，它将显示在下一张图像中。

I am new in python and I am using Quickstart: Extract printed text (OCR) using the REST API and Python in Computer Vision for text detection in Sales Fliers.So this algorithm is given has a coordinates Ymin, XMax, Ymin, and Xmax and draw a bounding boxes for each line of text, it show in this next image.

，但我想将附近的文本分组，以具有单个定界框架。因此对于上面的图片，它将有2个包含最接近的文本的边界框。

but I want to group the texts that are close by to have a single delimited frame. so for the case of the above image it will have 2 bounding boxes containing the closest text.

下面的代码提供了Ymin，XMax，Ymin和Xmax坐标，以及

The below code provide as a coordinates Ymin, XMax, Ymin, and Xmax and draw a bounding boxes for each line of text.

import requests
# If you are using a Jupyter notebook, uncomment the following line.
%matplotlib inline
import matplotlib.pyplot as plt
from matplotlib.patches import Rectangle
from PIL import Image
from io import BytesIO

# Replace <Subscription Key> with your valid subscription key.
subscription_key = "f244aa59ad4f4c05be907b4e78b7c6da"
assert subscription_key

vision_base_url = "https://westcentralus.api.cognitive.microsoft.com/vision/v2.0/"

ocr_url = vision_base_url + "ocr"

# Set image_url to the URL of an image that you want to analyze.
image_url = "https://cdn-ayb.akinon.net/cms/2019/04/04/e494dce0-1e80-47eb-96c9-448960a71260.jpg"

headers = {'Ocp-Apim-Subscription-Key': subscription_key}
params  = {'language': 'unk', 'detectOrientation': 'true'}
data    = {'url': image_url}
response = requests.post(ocr_url, headers=headers, params=params, json=data)
response.raise_for_status()

analysis = response.json()

# Extract the word bounding boxes and text.
line_infos = [region["lines"] for region in analysis["regions"]]
word_infos = []
for line in line_infos:
    for word_metadata in line:
        for word_info in word_metadata["words"]:
            word_infos.append(word_info)
word_infos

# Display the image and overlay it with the extracted text.
plt.figure(figsize=(100, 20))
image = Image.open(BytesIO(requests.get(image_url).content))
ax = plt.imshow(image)
texts_boxes = []
texts = []
for word in word_infos:
    bbox = [int(num) for num in word["boundingBox"].split(",")]
    text = word["text"]
    origin = (bbox[0], bbox[1])
    patch  = Rectangle(origin, bbox[2], bbox[3], fill=False, linewidth=3, color='r')
    ax.axes.add_patch(patch)
    plt.text(origin[0], origin[1], text, fontsize=2, weight="bold", va="top")
#     print(bbox)
    new_box = [bbox[1], bbox[0], bbox[1]+bbox[3], bbox[0]+bbox[2]]
    texts_boxes.append(new_box)
    texts.append(text)
#     print(text)
plt.axis("off")
texts_boxes = np.array(texts_boxes)
texts_boxes

输出边界框

array([[  68,   82,  138,  321],
       [ 202,   81,  252,  327],
       [ 261,   81,  308,  327],
       [ 364,  112,  389,  182],
       [ 362,  192,  389,  305],
       [ 404,   98,  421,  317],
       [  92,  421,  146,  725],
       [  80,  738,  134, 1060],
       [ 209,  399,  227,  456],
       [ 233,  399,  250,  444],
       [ 257,  400,  279,  471],
       [ 281,  399,  298,  440],
       [ 286,  446,  303,  458],
       [ 353,  394,  366,  429]]

但我想然后合并成近距离。

But I want to merge then by close distances.

推荐答案

谢谢@recnac，您的算法可以帮助我解决问题。

Thank you @recnac your algorithms help me to solved it.

我的解决方法是这样的。
生成一个新框，合并距离很近的文本框以获得一个新框。

My solution was this. Generate a new box,merge the text boxes with close distances for to get a new boxes. In which have a close texts.

#Distance definition  between text to be merge
dist_limit = 40

#Copy of the text and object arrays
texts_copied = copy.deepcopy(texts)
texts_boxes_copied = copy.deepcopy(texts_boxes)


#Generate two text boxes a larger one that covers them
def merge_boxes(box1, box2):
    return [min(box1[0], box2[0]), 
         min(box1[1], box2[1]), 
         max(box1[2], box2[2]),
         max(box1[3], box2[3])]



#Computer a Matrix similarity of distances of the text and object
def calc_sim(text, obj):
    # text: ymin, xmin, ymax, xmax
    # obj: ymin, xmin, ymax, xmax
    text_ymin, text_xmin, text_ymax, text_xmax = text
    obj_ymin, obj_xmin, obj_ymax, obj_xmax = obj

    x_dist = min(abs(text_xmin-obj_xmin), abs(text_xmin-obj_xmax), abs(text_xmax-obj_xmin), abs(text_xmax-obj_xmax))
    y_dist = min(abs(text_ymin-obj_ymin), abs(text_ymin-obj_ymax), abs(text_ymax-obj_ymin), abs(text_ymax-obj_ymax))

    dist = x_dist + y_dist
    return dist

#Principal algorithm for merge text 
def merge_algo(texts, texts_boxes):
    for i, (text_1, text_box_1) in enumerate(zip(texts, texts_boxes)):
        for j, (text_2, text_box_2) in enumerate(zip(texts, texts_boxes)):
            if j <= i:
                continue
            # Create a new box if a distances is less than disctance limit defined 
            if calc_sim(text_box_1, text_box_2) < dist_limit:
            # Create a new box  
                new_box = merge_boxes(text_box_1, text_box_2)            
             # Create a new text string 
                new_text = text_1 + ' ' + text_2

                texts[i] = new_text
                #delete previous text 
                del texts[j]
                texts_boxes[i] = new_box
                #delete previous text boxes
                del texts_boxes[j]
                #return a new boxes and new text string that are close
                return True, texts, texts_boxes

    return False, texts, texts_boxes


need_to_merge = True

#Merge full text 
while need_to_merge:
    need_to_merge, texts_copied, texts_boxes_copied = merge_algo(texts_copied, texts_boxes_copied)

texts_copied

这篇关于将边界框合并为一个的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！