如何验证网络摄像头校准的正确性? [英] How to verify the correctness of calibration of a webcam?
问题描述
我对相机校准技术完全陌生……我正在使用 OpenCV 棋盘技术……我正在使用 Quantum 的网络摄像头……
I am totally new to camera calibration techniques... I am using OpenCV chessboard technique... I am using a webcam from Quantum...
这是我的观察和步骤..
Here are my observations and steps..
- 我一直保持每个国际象棋的正方形边长 = 3.5 厘米.这是一个 7 x 5 棋盘，带有 6 x 4 内角.我在距离网络摄像头 1 到 1.5 m 的地方拍摄了总共 10 张不同视图/姿势的图像.
我正在按照 Bradski 的 Learning OpenCV 中的 C 代码进行校准.我的校准代码是
- I have kept each chess square side = 3.5 cm. It is a 7 x 5 chessboard with 6 x 4 internal corners. I am taking total of 10 images in different views/poses at a distance of 1 to 1.5 m from the webcam.
I am following the C code in Learning OpenCV by Bradski for the calibration. my code for calibration is
cvCalibrateCamera2(object_points,image_points,point_counts,cvSize(640,480),intrinsic_matrix,distortion_coeffs,NULL,NULL,CV_CALIB_FIX_ASPECT_RATIO);
在调用这个函数之前，我将内在矩阵的对角线上的第一个和第二个元素作为一个元素，以保持焦距的比率不变，并使用 CV_CALIB_FIX_ASPECT_RATIO
随着棋盘距离的变化fx和fy也在变化，fx:fy几乎等于1. cx 和 cy 值的顺序是 200 到 400. fx 和 fy 的顺序是当我改变距离时为 300 - 700.
With the change in distance of the chess board the fx and fy are changing with fx:fy almost equal to 1. there are cx and cy values in order of 200 to 400. the fx and fy are in the order of 300 - 700 when I change the distance.
目前我已经把所有的失真系数都设置为零，因为我没有得到好的结果，包括失真系数.我的原图比没变形的还帅！！
Presently I have put all the distortion coefficients to zero because I did not get good result including distortion coefficients. My original image looked handsome than the undistorted one!!
我是否正确进行了校准?.我应该使用 CV_CALIB_FIX_ASPECT_RATIO 以外的任何其他选项吗?如果有，是哪一个?
Am I doing the calibration correctly?. Should I use any other option than CV_CALIB_FIX_ASPECT_RATIO?. If yes, which one?
推荐答案
嗯，你在找帅"吗?还是准确"?
Hmm, are you looking for "handsome" or "accurate"?
相机校准是计算机视觉中为数不多的可以直接用物理术语量化精度并通过物理实验验证的主题之一.通常的教训是 (a) 你的数字和你投入的努力(和金钱)一样好，并且 (b) 真正的准确性(与想象相反)是昂贵的，所以你应该提前弄清楚什么您的应用程序确实需要精度.
Camera calibration is one of the very few subjects in computer vision where accuracy can be directly quantified in physical terms, and verified by a physical experiment. And the usual lesson is that (a) your numbers are just as good as the effort (and money) you put into them, and (b) real accuracy (as opposed to imagined) is expensive, so you should figure out in advance what your application really requires in the way of precision.
如果您查看甚至非常便宜的镜头/传感器组合(百万像素及以上)的几何规格，就会很明显，在桌面空间体积内理论上可以实现亚亚毫米校准精度.只需计算出(根据相机传感器的规格表)由一个像素构成的立体角 - 您会被钱包触手可及的空间分辨率所震撼.然而，要真正达到可重复的接近理论准确度的东西还需要努力.
If you look up the geometrical specs of even very cheap lens/sensor combinations (in the megapixel range and above), it becomes readily apparent that sub-sub-mm calibration accuracy is theoretically achievable within a table-top volume of space. Just work out (from the spec sheet of your camera's sensor) the solid angle spanned by one pixel - you'll be dazzled by the spatial resolution you have within reach of your wallet. However, actually achieving REPEATABLY something near that theoretical accuracy takes work.
以下是一些建议(来自个人经验)，可帮助您获得良好的自制设备校准体验.
Here are some recommendations (from personal experience) for getting a good calibration experience with home-grown equipment.
如果您的方法使用平面目标(棋盘"或类似的)，请制造一个好的目标.选择非常平坦的背衬(对于您提到的尺寸，5 毫米厚或更多的窗玻璃非常好，但显然易碎).验证其相对于另一个边缘(或更好的激光束)的平整度.在不会太容易拉伸的厚纸上打印图案.在粘合之前将其打印在背衬上并确认方形边确实非常接近正交.廉价的喷墨或激光打印机不是为严格的几何精度而设计的，不要盲目相信它们.最佳做法是使用专业的打印店(即使是 Kinko 打印机也会比大多数家用打印机做得更好).然后非常小心地将图案贴在背衬上，使用喷胶并用软布慢慢擦拭以避免气泡和拉伸.等待一天或更长时间，让胶水固化，胶纸应力达到长期稳定状态.最后测量拐角位置，用好的卡尺和放大镜.对于平均值"，您可能会得到一个单一的数字.正方形大小，但它必须是实际测量的平均值，而不是希望-n-祈祷的平均值.最佳做法是实际使用测量位置表.
If your method uses a flat target ("checkerboard" or similar), manufacture a good one. Choose a very flat backing (for the size you mention window glass 5 mm thick or more is excellent, though obviously fragile). Verify its flatness against another edge (or, better, a laser beam). Print the pattern on thick-stock paper that won't stretch too easily. Lay it after printing on the backing before gluing and verify that the square sides are indeed very nearly orthogonal. Cheap ink-jet or laser printers are not designed for rigorous geometrical accuracy, do not trust them blindly. Best practice is to use a professional print shop (even a Kinko's will do a much better job than most home printers). Then attach the pattern very carefully to the backing, using spray-on glue and slowly wiping with soft cloth to avoid bubbles and stretching. Wait for a day or longer for the glue to cure and the glue-paper stress to reach its long-term steady state. Finally measure the corner positions with a good caliper and a magnifier. You may get away with one single number for the "average" square size, but it must be an average of actual measurements, not of hopes-n-prayers. Best practice is to actually use a table of measured positions.
注意温度和湿度的变化:纸会从空气中吸收水分，背衬会膨胀和收缩.令人惊讶的是，您可以找到多少篇报告亚毫米校准精度的文章，而无需引用环境条件(或对它们的目标响应).不用说，他们大多是废话.与普通金属板相比，玻璃的温度膨胀系数较低，这是选择前者作为背衬的另一个原因.
Watch your temperature and humidity changes: paper adsorbs water from the air, the backing dilates and contracts. It is amazing how many articles you can find that report sub-millimeter calibration accuracies without quoting the environment conditions (or the target response to them). Needless to say, they are mostly crap. The lower temperature dilation coefficient of glass compared to common sheet metal is another reason for preferring the former as a backing.
不用说，您必须禁用相机的自动对焦功能，如果有的话:对焦会物理移动镜头内的一块或多块玻璃，从而改变(稍微)视场和(通常很多)镜头畸变和主要点.
Needless to say, you must disable the auto-focus feature of your camera, if it has one: focusing physically moves one or more pieces of glass inside your lens, thus changing (slightly) the field of view and (usually by a lot) the lens distortion and the principal point.
将相机放在不易振动的稳定支架上.根据应用程序的需要(不是校准 - 校准程序和目标必须根据应用程序的需要设计，而不是相反).甚至不要想事后触摸相机或镜头.如果可能的话，避免复杂"镜头 - 例如变焦镜头或广角镜头.例如，变形镜头需要的模型比 OpenCV 提供的现有模型复杂得多.
Place the camera on a stable mount that won't vibrate easily. Focus (and f-stop the lens, if it has an iris) as is needed for the application (not the calibration - the calibration procedure and target must be designed for the app's needs, not the other way around). Do not even think of touching camera or lens afterwards. If at all possible, avoid "complex" lenses - e.g. zoom lenses or very wide angle ones. For example, anamorphic lenses require models much more complex than stock OpenCV makes available.
进行大量测量和拍照.您希望每张图像有数百个测量值(角)和数十个图像.就数据而言，越多越好.10x10 棋盘格是我考虑的绝对最小值.我通常以 20x20 的分辨率工作.
Take lots of measurements and pictures. You want hundreds of measurements (corners) per image, and tens of images. Where data is concerned, the more the merrier. A 10x10 checkerboard is the absolute minimum I would consider. I normally worked at 20x20.
拍照时跨度校准音量.理想情况下，您希望您的测量结果均匀分布在您将使用的空间体积中.最重要的是，确保在某些图片中将目标相对于焦轴的角度显着 - 以校准您需要看到"目标的焦距.一些真实的透视缩短.为获得最佳效果，请使用可重复的机械夹具移动目标.一个好的是单轴转台，它会给你一个很好的目标运动先验模型.
Span the calibration volume when taking pictures. Ideally you want your measurements to be uniformly distributed in the volume of space you will be working with. Most importantly, make sure to angle the target significantly with respect to the focal axis in some of the pictures - to calibrate the focal length you need to "see" some real perspective foreshortening. For best results use a repeatable mechanical jig to move the target. A good one is a one-axis turntable, which will give you an excellent prior model for the motion of the target.
在拍照时尽量减少振动和相关的运动模糊.
Minimize vibrations and associated motion blur when taking photos.
使用良好的照明.真的.令人惊讶的是，我经常看到人们在游戏后期意识到你需要大量的光子来校准相机:-) 使用漫射环境光，并将其从视野两侧的白卡上反射.
Use good lighting. Really. It's amazing how often I see people realize late in the game that you need a generous supply of photons to calibrate a camera :-) Use diffuse ambient lighting, and bounce it off white cards on both sides of the field of view.
观察角点提取代码在做什么.在图像上绘制检测到的角位置(例如在 Matlab 或 Octave 中)，并判断它们的质量.使用严格的阈值及早去除异常值比信任捆绑调整代码中的稳健器要好.
Watch what your corner extraction code is doing. Draw the detected corner positions on top of the images (in Matlab or Octave, for example), and judge their quality. Removing outliers early using tight thresholds is better than trusting the robustifier in your bundle adjustment code.
如果可以，请约束您的模型.例如，如果您没有充分的理由相信您的镜头明显偏离图像中心，请不要尝试估计主要点，只需在第一次尝试时将其固定在图像中心即可.主要点的位置通常很难观察到，因为它本质上与非线性失真的中心和 平行于目标到图像平面的分量相混淆.相机的翻译.让它正确需要一个精心设计的程序，它产生三个或更多场景的独立消失点和非常好的非线性失真包围.同样，除非您有理由怀疑镜头焦轴确实倾斜了 w.r.t.传感器平面，将相机矩阵的 (1,2) 分量固定为零.一般来说，使用满足您的测量和您的应用程序需求的最简单的模型(这是您的奥卡姆剃刀).
Constrain your model if you can. For example, don't try to estimate the principal point if you don't have a good reason to believe that your lens is significantly off-center w.r.t the image, just fix it at the image center on your first attempt. The principal point location is usually poorly observed, because it is inherently confused with the center of the nonlinear distortion and by the component parallel to the image plane of the target-to-camera's translation. Getting it right requires a carefully designed procedure that yields three or more independent vanishing points of the scene and a very good bracketing of the nonlinear distortion. Similarly, unless you have reason to suspect that the lens focal axis is really tilted w.r.t. the sensor plane, fix at zero the (1,2) component of the camera matrix. Generally speaking, use the simplest model that satisfies your measurements and your application needs (that's Ockam's razor for you).
当您从优化器获得具有足够低 RMS 误差(通常为十分之几像素，另请参见 Josh 的回答)的校准解决方案时，绘制残差的 XY 模式(对于所有图像中的每个角)并看看它是否是一个以 (0, 0) 为中心的圆形云.团块"异常值或残差云的不圆度正在敲响警钟，表明某些事情非常错误 - 可能是由于角点检测或匹配不当或镜头失真模型不合适而导致的异常值.
When you have a calibration solution from your optimizer with low enough RMS error (a few tenths of a pixel, typically, see also Josh's answer below), plot the XY pattern of the residual errors (predicted_xy - measured_xy for each corner in all images) and see if it's a round-ish cloud centered at (0, 0). "Clumps" of outliers or non-roundness of the cloud of residuals are screaming alarm bells that something is very wrong - likely outliers due to bad corner detection or matching, or an inappropriate lens distortion model.
拍摄额外的图像来验证解决方案的准确性 - 使用它们来验证镜头畸变实际上已消除，并且校准模型预测的平面单应性实际上与从测量角恢复的平面单应性相匹配.
Take extra images to verify the accuracy of the solution - use them to verify that the lens distortion is actually removed, and that the planar homography predicted by the calibrated model actually matches the one recovered from the measured corners.
这篇关于如何验证网络摄像头校准的正确性?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
