分析手绘流程图 [英] Analyzing a hand-drawn flowchart diagram

问题描述

我正在尝试检测手绘图中的对象和文本.

I'm trying to detect objects and text in a hand-drawn diagram.

我的目标是能够解析"类似

My goal is to be able to "parse" something like this into an object structure for further processing. My first aim is to detect text, lines and boxes (arrows etc... are not important (for now ;))

我能做的扩张，侵蚀，大津阈值，反转等，容易得到类似的此

I can do Dilatation, Erosion, Otsu thresholding, Invert etc and easily get to something like this

我需要一些指导的是后续步骤. 我有几个想法:

What I need some guidance for are the next steps. I've have several ideas:

1. 轮廓分析
2. 使用UNIPEN的OCR
3. 边缘检测

轮廓分析

我一直在阅读CodeProject上的"C#中用于图像识别的轮廓分析"，这可能是识别盒子等的好方法.但是我的问题是盒子是连接的，因此没有形成单独的对象来匹配模板. 因此，如果这是可行的方法，我需要一些建议.

I've been reading about "Contour Analysis for Image Recognition in C#" on CodeProject which could be a great way to recognize boxes etc. but my issue is that the boxes are connected and therefore do not form separate objects to match with a template. Therefore I need some advises IF this is a feasible way to go.

使用UNIPEN的OCR

我想使用UNIPEN(请参阅CodeProject上的使用多神经网络的大型模式识别系统")识别手写字母，然后从图像中删除"它们，仅留下方框和线条.

I would like to use UNIPEN (see "Large pattern recognition system using multi neural networks" on CodeProject) to recognize handwritten letters and then "remove" them from the image leaving only the boxes and lines.

边缘检测 另一种方法是检测所有线条和拐角，然后推断图像所组成的方框和线条.在这种情况下，有关如何拉直直线并找到90度角的想法将很有帮助.

Edge detection Another way could be to detect all lines and corners and in that way infer the boxes and lines that the image consist of. In that case ideas on how to straighten the lines and find the 90 degree corners would be helpful.

一般来说，我认为我只需要一些指针就可以应用哪种策略，而不是代码示例(尽管会很棒；))

Generally, I think I just need some pointers on which strategy to apply, not code samples (though it would be great ;))

推荐答案

我将尝试回答有关轮廓分析及其之间的线的问题.

I will try to answer about the contour analysis and the lines between them.

如果需要将互连的盒子变成单独的对象，可以很容易地实现:

If you need to turn the interconnected boxes into separate objects, that can be achieved easily enough:

1. 通过形态学闭合来闭合盒子边缘的空隙
2. 执行连接组件的标签并查找紧凑的对象(例如，其面积接近其边界框区域的对象)

您将获得盒子的内部.这些可以是椭圆形或矩形，也可以是您在普通图中发现的任何形状，轮廓分析可以告诉您哪种形状.封闭的背景区域(例如，示例图中ABC链接之间的空间)可能会出现问题.您可以根据其边界框与其他多个对象的边界框重叠的标准来消除它们.

You will get the insides of the boxes. These can be elliptical or rectangular or any shape you may find in common diagrams, the contour analysis can tell you which. A problem may arise for enclosed background areas (e.g. the space between the ABC links in your example diagram). You might eliminate these on the criterion that their bounding box overlaps with multiple other objects' bounding boxes.

现在使用HoughLinesP查找线段.如果段在距其中一个对象的边缘一定距离内完成或开始，则可以假定该段已连接到该对象.

Now find line segments with HoughLinesP. If a segment finishes or starts within a certain distance of the edge of one of the objects, you can assume it is connected to that object.

作为附加的触摸，您可以尝试通过检查线段端点附近的宽度轮廓来检测两侧的箭头末端.

As an added touch you could try to detect arrow ends on either side by checking the width profile of the line segments in a neighbourhood of their endpoints.

这是一个有趣的问题，我会尽力记住它，并将其交给我的学生，以磨砺自己的牙齿.

It is an interesting problem, I will try to remember it and give it to my students to grit their teeth on.

这篇关于分析手绘流程图的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！