如何识别图像中的球，然后在 3D 中对其进行建模(HawkEye 系统) [英] How to recognize a ball in an image and then model it in 3D(HawkEye System)

问题描述

我们正在尝试为我们的大学项目开发​​用于板球的 HawkEye 系统.HawkEye系统中使用的流程如下:

We are trying to develop HawkEye System used in cricket for our college project. The process used in HawkEye System is as follows:

1. 需要从投球手的手到击球手的不同时间(不同点)的球图像(在球的整个飞行过程中)
2. 确定球在整个飞行过程中不同时间点的 (x,y) 坐标
3. 将(x,y)坐标转换成对应的3D坐标(x,y,z)
4. 模拟球在整个飞行过程中的轨迹以及球的周围环境，包括场地、球场、小门、体育场
5. 延长球的轨迹，看看球是否会击中小门

到目前为止，我们计划完成这个项目:

So far this is what we've planned to accomplish this project:

我们将从腿裁判的位置拍摄击球手的视频，然后在 vlc player 中以慢动作播放该视频，同时拍摄球飞行的多个屏幕截图，我想这将完成第 1 步.

we'll shoot the video of the batsman from the leg umpire's position and then play that video in slow motion in vlc player and simultaneously taking multiple screenshots of the flight of the ball, i guess this will take care of the step 1.

但是现在我们被困在了第 2 步，我们现在面临的问题是如何识别和找到特定实例中球的 (x,y) 坐标(从拍摄的球的图像中)从腿侧)如果我们可以找到球的 (x,y) 并且如果相机到某个参考点的距离是已知的，那么我们可以找到图像的深度，即 z 坐标，因此我们可以找到相应的 (x,y,z) 坐标，然后我们可以使用 OpenGL 在 3D 中对其进行建模

but right now we are stuck in step 2, the problem which we're facing now is that how to recognize and find the (x,y) coordinate of the ball at a particular instance (from the image of the ball taken from leg side) if we can find the (x,y) of the ball and if the distance of the camera from some reference point is known then we can find the depth of the image i.e. the z-coordinate, hence we can find out the corresponding (x,y,z) coordinates and then we can model it in 3D using OpenGL

我们正在尝试用 C++ 实现它

we're trying to implement it in C++

感谢任何帮助:)

快速
我开始知道，在真正的 HawkEye System 中，6 个摄像头是在板球场的圆周上调整的，所有摄像头都以 60 度角分开，HawkEye 仅使用 4 个摄像头就可以完美工作，但为了提高精度，还使用了 2 个额外的摄像头.由于我们没有那么多摄像机，我想我们将使用 3 个摄像机，它们保持在以 120 度分隔的场地圆周上，为了降低复杂性，我们将选择一个半径 = 5m 的小场地，但是我们不确定将相机放在哪里以获得更准确的结果，可能位置可能是:一个在腿边，一个在越位，第三个在正前方，但我仍然不确定应该放在什么位置选择

A quick edit:
I came to know that in real HawkEye System 6 cameras are adjusted on the circumference of the cricket field, all the cameras are seperated by an angle of 60 degree, HawkEye can work perfectly using 4 cameras only but for better precision 2 extra cameras are used. since we dont have so many cameras, I think we'll be using 3 cameras kept on the circumference of the field seperated by 120 degrees and to reduce the complexity we'll be choosing a small field may be one with radius=5m, but we're not sure where to place the cameras to get more accurate results, may be the positions can be: one on the legside, one on off-side and the third one straight in front but i'm still not sure what positions to choose

这种方法称为多相机校准，对于球识别，我认为我们应该选择 OpenCV 而不是 MATLAB，因为 OpenCV 完成的图像处理速度更快

this approach is called Multi Camera Calibration and for ball recognition I think we should choose OpenCV over MATLAB because of more speedy image processing done by OpenCV

大家有什么想说的吗?

推荐答案

关于第 2 步，提取球的位置，有多种可能的方法和文献来源.我强烈建议您查看关于机器人足球 (Robocup) 的工作，其中包含许多类似问题的示例.

With respect to step 2, extracting the location of a ball, there are a multitude of possible approaches and sources of literature. I would strongly recommend looking into the work on Robot soccer (Robocup), which contains many examples of similar problems.

在理想的世界中(比如白底黑盘)，起点可能是使用类似 霍夫变换，或轮廓跟踪，并使用合成轮廓的统计矩提取位置.

In an ideal world (say a black disk on a white background), the starting point would probably be to use something like a Hough Transform, or contour tracing, and extracting the position using statistical moments of the resultant contour.

这种方法的挑战在于，板球场肯定会具有难以移除的背景特征.通过反复试验，您可以使用常见的图像处理技术，例如背景减法、形态学算子、边缘检测器、颜色过滤和阈值处理，以提高您始终如一地找到球的能力.根据过去的经验，我强烈建议您使用一组工具来快速构建图像处理管道和技术的原型，可能是 MATLAB.

The challenge of this approach is that a cricket field is most definitely going to have background features that are challenging to remove. With some trial and error you may be able to use common image processing techniques such as background subtraction, morphological operators, edge detectors, color filtering and thresholding to improve your capability to consistently find the ball. From past experience, I strongly recommend using a set of tools that allows you to rapidly prototype image processing pipelines and techniques, probably MATLAB.

也许更可靠的表述这个问题的方式是，如果你对球之前的位置有一些了解，那么你可以合理地估计在小球之后球应该在哪里时间.这是最优估计领域，以及卡尔曼滤波器.一个很好的介绍性文本，尽管来自一个非常不同的问题空间，是 Thrun 等人的概率机器人.

Perhaps a more robust way to phrase this problem, leading into the following sections, is that if you have some idea of where the ball was previously, then you can make a reasonable estimation of where the ball should be after some small amount of time. This is the field of optimal estimation, and Kalman Filters. A good introductory text, albeit from a very different problem space, is Probabilistic Robotics by Thrun et al.

这篇关于如何识别图像中的球，然后在 3D 中对其进行建模(HawkEye 系统)的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！