Python校准相机 [英] Python calibrate camera
问题描述
我有以下图像I1.我没有抓住它.我是从Google下载的
I have the following image I1. I did not capture it. I downloaded it from Google
我将已知的单应性H应用于I1，以获得以下图像I2.
I apply a known homography H to I1 to obtain the following image I2.
我想假设照相机已经拍摄了以上I2的照片.我不知道这台相机的相机矩阵，所以我想找到它.要找到此相机矩阵mtx，我使用的是OpenCV相机校准方法:ret, mtx, dist, rvecs, tvecs = cv2.calibrateCamera(objpoints, imgpoints, gray.shape[::-1], None, None,flags=cv2.CALIB_FIX_ASPECT_RATIO|cv2.CALIB_FIX_K1|cv2.CALIB_FIX_K2|cv2.CALIB_FIX_K3|cv2.CALIB_FIX_K4|cv2.CALIB_FIX_K5)
I want to assume that a camera has taken this above shot of I2. I do not know the camera matrix of this camera and I want to find it. To find this camera matrix mtx, I am using the OpenCV camera calibration method: ret, mtx, dist, rvecs, tvecs = cv2.calibrateCamera(objpoints, imgpoints, gray.shape[::-1], None, None,flags=cv2.CALIB_FIX_ASPECT_RATIO|cv2.CALIB_FIX_K1|cv2.CALIB_FIX_K2|cv2.CALIB_FIX_K3|cv2.CALIB_FIX_K4|cv2.CALIB_FIX_K5)
这是使用正方形完成的，它是真实世界和图像坐标.我选择图像I1中的一个正方形，并使用单应性H来获取I2中该正方形的对应角点.由于我知道I2中的这些对应点形成一个正方形，因此我应该能够从这些点获取相机矩阵.但是，当我在图像的不同位置拍摄相同的正方形时，会得到不同的相机矩阵.为什么是这样?我在做什么错，我该如何解决?如何计算正确的相机矩阵?
This is done using a square and it's real world and image coordinates. I choose a square in image I1 and get the corresponding corner points of the square in I2 using homography H. Since I know that these corresponding points in I2 form a square, I should be able to get the camera matrix from these points. However, when I take the same square at a different location in the image, I get a different camera matrix. Why is this? What am I doing wrong and how I can I fix it? How do I calculate the correct camera matrix?
下面显示一个示例.对于这两个选定的正方形，我从calibrateCamera函数获得了不同的mtx值.
An example is shown below. For these two chosen squares, I get different values of mtx from the calibrateCamera function.
注意:上图中的红色点不是I1中完美正方形的角点.我刚刚对它们进行了粗略标记，以表达我的观点，即当我拍摄两个相同大小但在不同位置的正方形时，相机矩阵的值将有所不同.
NOTE: The red points in the above images are not the corner points of a perfect square in I1. I have just roughly marked them to convey my point that when I take two squares of the same size but at different locations, I get different values for the camera matrix.
推荐答案
这是一个好问题，涉及校准和计算几何的几个重要问题.我将提供一个深入的答案，希望我能使这些事情变得清楚.
This is a good question which involves several important issues with calibration and computational geometry. I'm going to provide an in-depth answer that I hope will make these things clear.
执行相机校准时，如果使用不同的对应关系重复校准，则会有三个原因，您可以拥有不同的固有矩阵.
When performing camera calibration there are three reasons why you can have different intrinsic matrices, if you repeat the calibration using different sets of correspondences.
- 信件很吵.
- 相机校准问题未完全解决.这意味着没有足够的对应信息来唯一地解析所有相机参数.
- 相机校准使用不精确或过于严格的相机模型.
原因1 应该很明显.如果对应关系因测量噪声而损坏，则使用不同的对应关系集通常会获得不同的校准.这是因为在校准过程中，相机是一个优化过程，其中相机参数被优化为最适合对应关系.当有噪音时，最佳配合可能会因测得的噪音而异.
Reason 1 should be fairly obvious. If the correspondences are corrupted by measurement noise, then you will generally obtain different calibrations if you use different sets of correspondences.This is because during calibration thee is an optimization process where the camera parameters are optimized to best fit the correspondences. When there is noise, the best fit can vary depending on the measured noise.
原因2 .例如，如果每个图像只有三个对应关系，则校准问题正在确定中.您可以通过计算参数来考虑这一点.三种对应关系在校准方程式上提供了6个约束(通过x和y位置，每种对应关系对应两个).现在，当我们进行校准时，我们必须共同估算校准对象的姿态(每个图像具有6个自由度)，再加上内在函数的未知数(焦距，主点，失真等).因此，未知数多于约束，因此许多校准可以无限！因此，如果您选择了三种不同的对应关系集，则返回校准(如果一个返回了)将永远是不正确的，并且通常也不会是相同的.
Reason 2 happens if you try to calibrate using insufficient information. For example, if you only had three correspondences per image, the calibration problem is under determined. You can think of this through counting parameters. Three correspondences provides 6 constraints on the calibration equations (two for each correspondence through it's x and y position). Now, when we calibrate we must jointly estimate the pose of the calibration object (which has 6 degrees of freedom per image), plus the unknowns for the intrinsics (focal length, principal point, distortion etc.). There are therefore more unknowns than constraints, so there can be infinity many calibrations! If therefore you chose different sets of three correspondences, the returns calibration (if one is returned at all) will never be correct and generally never be the same.
原因3 更加微妙.为了解释这一点，请记住，可以通过指定具有不同数量的未知固有参数的摄像机来进行校准.在校准信息非常有限的情况下，减少未知数的数量通常是很好的.例如，如果使用单个图像进行校准，则平面校准对象将为您在校准时为每个图像最多提供8个约束(因为单应性具有8个自由度).要获得飞机的姿势，需要6个约束，因此每个图像剩下2个约束.如果只有一幅图像，则当存在两个以上的未知数(例如焦距和镜头畸变)时，您将无法进行校准.因此，如果要使用单个图像进行校准，则必须减少未知数.
Reason 3 is more subtle. To explain this, remember that calibration can be done by specifying a camera with different numbers of unknown intrinsic parameters. It is often good to reduce the number of unknowns in cases where you have very limited calibration information. For example, if calibrating with a single image, a planar calibration object will give you a maximal of 8 constraints per image on the calibration (because a homography has 8 degrees of freedom). 6 constraints are required to get the plane's pose, so we are left with 2 remaining constraints per image. If you only have a single image, you cannot do a calibration when there are more than 2 unknowns (e.g. focal lengths and lens distortion). Therefore if we wanted to calibrate using a single image we must reduce unknowns.
您的情况 在您的情况下，您已将未知数减少到单焦距(f = fx = fy)和相机的主要点.这是3个未知数，但要记住，对单个图像进行校准意味着您最多只能有2个固有未知数.因此，您有一个约束不足的问题(请参见上面的原因2).
What's happening in your case In your case you have reduced the unknowns to a single focal length (f=fx=fy) and the camera's principal point. That's 3 unknowns, but recall that to do calibration with a single image means you can only have a maximum of 2 intrinsic unknowns. Therefore you have an under-constrained problem (see reason 2 above).
现在，您可以决定通过将主点固定到图像中心来克服此问题，这是很常见的事情，因为它通常是真实主点的良好近似值.现在，您有一个校准问题，其中有1个未知的固有(f).重要的问题是，如果我们尝试使用单个图像和4个无噪音的对应关系来校准f，那么我们可以期望使用不同的对应关系集获得相同的值吗?您可能会认为是，但答案是否定的.
Now, you might decide to overcome this by fixing the principal point to the image centre, which is a common thing to do as it is often a good approximation for the real principal point. Now you have a calibration problem with 1 unknown intrinsic (f). The important question is, if we try to calibrate f using a single image and 4 noiseless correspondences, can we expect to get the same value using different sets of correspondences? You might think yes but the answer is no.
原因是因为校准过程将解决一个过度约束的问题(8个约束和7个未知数).通常，它将使用功能最小化过程来解决此问题(就像OpenCV的calibrateCamera方法一样).在OpenCV中，这是通过最小化重新投影误差来完成的.解决方案将根据您提供的信件而有所不同.很难想象，因此请考虑一个不同的问题，在该问题中，您试图将一条直线拟合到一条稍微弯曲的线上的点.直线是数据的过度简化模型.如果我们尝试通过对曲线上的两个点进行采样来拟合直线，则最佳拟合解决方案将根据采样的点而变化.
The reason is because the calibration process will be solving an over-constrained problem (8 constraints and 7 unknowns). It will generally solve this (as OpenCV's calibrateCamera method does) using a function minimization process. In OpenCV, it is done by minimizing the reprojection error. The solution to this will vary depending on the correspondences you provide. This is rather tricky to imagine, so consider a different problem, where you're trying to fit a straight line to points on a slightly curved line. The straight line is an overly simplified model for the data. If we try to fit the line to the curved data, by sampling two points from it, the best fitting solution will change depending on which points are sampled.
在特定情况下，您可以通过使用恰好具有2个未知数的内在矩阵，删除标记以固定宽高比以及将主点固定在图像中心，来消除问题2和3.
In your particular case you can eliminate problems 2 and 3 by using an intrinsic matrix with exactly 2 unknowns, by removing the flag to fix the aspect ratio, and by fixing the principal point to the image centre.
这篇关于Python校准相机的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
