如何从图像中提取边缘强度不同的矩形? [英] How to extract rectangles of varying edge intensity from images?

问题描述

我正在尝试从支票图像中提取帐号.我所拥有的逻辑是，我试图找到包含帐号的矩形，对边界矩形进行切片，然后将其送入OCR以从中获取文本.

I am trying to extract the account number from an image of a cheque. The logic that I have is that, I am trying to find the rectangle that contains the account number, slice the bounding rectangle and then feed the slice into an OCR to get the text out of it.

我面临的问题是，当矩形不是很突出且颜色浅时，由于边缘未完全连接，我无法获得矩形轮廓.

The problem I am facing is when the rectangle is not very prominent and light colour, I am not able to get the rectangle contour since the edges are not connected totally.

如何克服这个问题? 我尝试过但没用的东西

How to overcome this? Things I tried, but did not work are

1. 我不能增加腐蚀迭代，以进一步腐蚀它，因为然后边缘与周围的黑色像素相连并形成不同的形状.
2. 降低阈值偏移量可能会有所帮助，但是效率似乎很低.由于代码必须处理几种类型的图像.我可以从偏移量10开始，并继续增加偏移量并检查是否找到矩形.对于突出显示的矩形(在偏移量20或更大时效果很好)的检查，这将大大增加检查时间.而且由于我没有条件检查矩形的边缘是否突出，因此必须在所有支票中应用循环.

请牢记以上几点.有人可以帮我解决这个问题吗?

Keeping the above points in mind. Can someone help me out with a solution to this problem?

使用的库和版本

scikit-image==0.13.1
opencv-python==3.3.0.10

代码

from skimage.filters import threshold_adaptive, threshold_local
import cv2

第1步:

image = cv2.imread('cropped.png')

第2步:

使用skimage中的自适应阈值来删除背景，以便获得帐号矩形框.这对于矩形更明显的检查效果很好，但是当矩形边缘较薄或颜色较浅时，阈值将导致 未连接的边缘，因此无法找到轮廓.我在问题的后面附加了一些示例.

Using adaptive threshold from skimage to remove the background, so that I can get the account number rectangle box. This works fine for the cheques where the rectangle is more pronounced, but when the rectangle edges are thin, or are lighter in colour, the threshold results in unconnected edges, because of which I am not able to find the contours. I have attached examples of this further down in the question.

account_number_block = cv2.cvtColor(image, cv2.COLOR_RGB2GRAY)
account_number_block = threshold_adaptive(account_number_block, 251, offset=20)
account_number_block = account_number_block.astype("uint8") * 255

第3步:

稍微侵蚀图像以尝试连接边缘的细小断开连接

Erode the image a bit to try to connect small disconnections in the edges

kernel = np.ones((3,3), np.uint8)

account_number_block = cv2.erode(account_number_block, kernel, iterations=5)

找到轮廓

(_, cnts, _) = cv2.findContours(account_number_block.copy(), cv2.RETR_LIST, cv2.CHAIN_APPROX_SIMPLE)
# cnts = sorted(cnts, key=cv2.contourArea)[:3]
rect_cnts = [] # Rectangular contours 
for cnt in cnts:
    approx = cv2.approxPolyDP(cnt,0.01*cv2.arcLength(cnt,True),True)
    if len(approx) == 4:
        rect_cnts.append(cnt)


rect_cnts = sorted(rect_cnts, key=cv2.contourArea, reverse=True)[:1]

工作示例

第1步:原始图像

第2步:阈值去除背景后.

Step 2: After thresholding to remove the background.

第3步:找到轮廓以找到帐号的矩形框.

Step 3: Finding contours to find rectangle box of the account number.

失败的工作示例-浅矩形边界.

第1步:读取原始图像

第2步:阈值消除背景后.请注意，矩形的边缘未连接，因此无法从中获取轮廓.

Step 2: After thresholding to remove the background. Notice that the edges of the rectangle are not connected, because of which I am not able to get the contour out of it.

第3步:找到轮廓以找到帐号的矩形框.

Step 3: Finding contours to find rectangle box of the account number.

推荐答案

import numpy as np
import cv2
import pytesseract as pt
from PIL import Image


#Run Main
if __name__ == "__main__" :

    image = cv2.imread("image.jpg", -1)

    # resize image to speed up computation
    rows,cols,_ = image.shape
    image = cv2.resize(image, (np.int32(cols/2),np.int32(rows/2)))

    # convert to gray and binarize
    gray_img = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
    binary_img = cv2.adaptiveThreshold(gray_img, 255, cv2.ADAPTIVE_THRESH_MEAN_C, cv2.THRESH_BINARY, 9, 9)

    # note: erosion and dilation works on white forground
    binary_img = cv2.bitwise_not(binary_img)

    # dilate the image to fill the gaps
    kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (3,3))
    dilated_img = cv2.morphologyEx(binary_img, cv2.MORPH_DILATE, kernel,iterations=2)

    # find contours, discard contours which do not belong to a rectangle
    (_, cnts, _) = cv2.findContours(dilated_img, cv2.RETR_LIST, cv2.CHAIN_APPROX_SIMPLE)
    rect_cnts = [] # Rectangular contours 
    for cnt in cnts:
        approx = cv2.approxPolyDP(cnt,0.01*cv2.arcLength(cnt,True),True)
        if len(approx) == 4:
            rect_cnts.append(cnt)

    # sort contours based on area
    rect_cnts = sorted(rect_cnts, key=cv2.contourArea, reverse=True)[:1]

    # find bounding rectangle of biggest contour
    box = cv2.boundingRect(rect_cnts[0])
    x,y,w,h = box[:]

    # extract rectangle from the original image
    newimg = image[y:y+h,x:x+w]

    # use 'pytesseract' to get the text in the new image
    text = pt.image_to_string(Image.fromarray(newimg))
    print(text)

    cv2.namedWindow('Image', cv2.WINDOW_NORMAL)
    cv2.imshow('Image', newimg)
    cv2.waitKey(0)
    cv2.destroyAllWindows()

结果: 03541140011724

结果: 34785736216

这篇关于如何从图像中提取边缘强度不同的矩形?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！