如何识别图像中的球，然后在 3D 中建模(HawkEye 系统) [英] How to recognize a ball in an image and then model it in 3D(HawkEye System)

问题描述

我们正在尝试为我们的大学项目开发​​用于板球的 HawkEye 系统.HawkEye系统中使用的流程如下:

We are trying to develop HawkEye System used in cricket for our college project. The process used in HawkEye System is as follows:

1. 需要从投球手的手到击球手(在球的整个飞行过程中)的不同时间点(不同点)的球图像
2. 确定球在整个飞行过程中不同时间点的 (x,y) 坐标
3. 将 (x,y) 坐标转换为对应的 3D 坐标 (x,y,z)
4. 模拟球在整个飞行过程中的轨迹以及球的周围环境，包括场地、球场、三柱门、体育场
5. 延长球的轨迹以查看球是否会击中三柱门

到目前为止，我们计划完成这个项目:

So far this is what we've planned to accomplish this project:

我们将从腿部裁判员的位置拍摄击球手的视频，然后在 vlc 播放器中以慢动作播放该视频，同时拍摄球飞行的多个屏幕截图，我想这将处理第 1 步.

we'll shoot the video of the batsman from the leg umpire's position and then play that video in slow motion in vlc player and simultaneously taking multiple screenshots of the flight of the ball, i guess this will take care of the step 1.

但是现在我们被困在第 2 步，我们现在面临的问题是如何识别和找到球在特定实例的 (x,y) 坐标(从拍摄的球的图像从腿侧)如果我们可以找到球的 (x,y) 并且如果相机与某个参考点的距离已知，那么我们可以找到图像的深度，即 z 坐标，因此我们可以找到对应的 (x,y,z) 坐标，然后我们可以使用 OpenGL 对其进行 3D 建模

but right now we are stuck in step 2, the problem which we're facing now is that how to recognize and find the (x,y) coordinate of the ball at a particular instance (from the image of the ball taken from leg side) if we can find the (x,y) of the ball and if the distance of the camera from some reference point is known then we can find the depth of the image i.e. the z-coordinate, hence we can find out the corresponding (x,y,z) coordinates and then we can model it in 3D using OpenGL

我们正在尝试用 C++ 实现它

we're trying to implement it in C++

任何帮助表示赞赏:)

快速
我才知道，在真实的 HawkEye System 6 摄像头是在板球场的圆周上调整的，所有摄像头都以 60 度角分开，HawkEye 仅使用 4 个摄像头即可完美工作，但为了获得更好的精度，需要额外使用 2 个摄像头.因为我们没有那么多相机，我想我们将使用 3 个相机保持在场地圆周上，相隔 120 度，为了降低复杂性，我们将选择一个半径 = 5m 的小场地，但是我们不确定将相机放置在哪里以获得更准确的结果，可能位置可以是:一个在腿侧，一个在越位，第三个在前面，但我仍然不确定要放置什么位置选择

A quick edit:
I came to know that in real HawkEye System 6 cameras are adjusted on the circumference of the cricket field, all the cameras are seperated by an angle of 60 degree, HawkEye can work perfectly using 4 cameras only but for better precision 2 extra cameras are used. since we dont have so many cameras, I think we'll be using 3 cameras kept on the circumference of the field seperated by 120 degrees and to reduce the complexity we'll be choosing a small field may be one with radius=5m, but we're not sure where to place the cameras to get more accurate results, may be the positions can be: one on the legside, one on off-side and the third one straight in front but i'm still not sure what positions to choose

这种方法称为多相机校准，对于球识别，我认为我们应该选择 OpenCV 而不是 MATLAB，因为 OpenCV 可以更快地完成图像处理

this approach is called Multi Camera Calibration and for ball recognition I think we should choose OpenCV over MATLAB because of more speedy image processing done by OpenCV

大家有什么想说的吗?

推荐答案

关于第2步，提取球的位置，有多种可能的方法和文献来源.我强烈建议您研究有关机器人足球 (Robocup) 的工作，其中包含许多类似问题的示例.

With respect to step 2, extracting the location of a ball, there are a multitude of possible approaches and sources of literature. I would strongly recommend looking into the work on Robot soccer (Robocup), which contains many examples of similar problems.

在理想的世界中(比如白色背景上的黑色磁盘)，起点可能是使用类似 霍夫变换，或轮廓跟踪，并使用所得轮廓的统计矩提取位置.

In an ideal world (say a black disk on a white background), the starting point would probably be to use something like a Hough Transform, or contour tracing, and extracting the position using statistical moments of the resultant contour.

这种方法的挑战在于，板球场肯定会具有难以去除的背景特征.通过一些尝试和错误，您可能能够使用常见的图像处理技术，例如背景减法、形态算子、边缘检测器、颜色过滤和阈值处理，以提高您始终如一地找到球的能力.根据过去的经验，我强烈建议您使用一组工具，这些工具可以让您快速制作图像处理管道和技术的原型，可能是 MATLAB.

The challenge of this approach is that a cricket field is most definitely going to have background features that are challenging to remove. With some trial and error you may be able to use common image processing techniques such as background subtraction, morphological operators, edge detectors, color filtering and thresholding to improve your capability to consistently find the ball. From past experience, I strongly recommend using a set of tools that allows you to rapidly prototype image processing pipelines and techniques, probably MATLAB.

也许更稳健的方式来表述这个问题，导致以下部分，如果您对球之前的位置有所了解，那么您可以在少量之后对球应该在哪里做出合理的估计的时间.这是最优估计领域，卡尔曼滤波器.Thrun 等人的概率机器人.

Perhaps a more robust way to phrase this problem, leading into the following sections, is that if you have some idea of where the ball was previously, then you can make a reasonable estimation of where the ball should be after some small amount of time. This is the field of optimal estimation, and Kalman Filters. A good introductory text, albeit from a very different problem space, is Probabilistic Robotics by Thrun et al.

这篇关于如何识别图像中的球，然后在 3D 中建模(HawkEye 系统)的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！