用opencv进行外在矩阵计算 [英] extrinsic matrix computation with opencv
问题描述
我正在使用opencv校准我的网络摄像头.因此，我所做的就是将网络摄像头固定在钻机上，以使其保持静态，并且我使用了棋盘校准图案并将其移动到相机前面，并使用检测到的点来计算校准.因此，正如我们在许多opencv示例中所发现的那样( https://docs.opencv.org/3.1.0/dc/dbb/tutorial_py_calibration.html )
I am using opencv to calibrate my webcam. So, what I have done is fixed my webcam to a rig, so that it stays static and I have used a chessboard calibration pattern and moved it in front of the camera and used the detected points to compute the calibration. So, this is as we can find in many opencv examples (https://docs.opencv.org/3.1.0/dc/dbb/tutorial_py_calibration.html)
现在，这给了我相机固有矩阵以及旋转和平移组件，用于将这些棋盘视图中的每一个从棋盘空间映射到世界空间.
Now, this gives me the camera intrinsic matrix and a rotation and translation component for mapping each of these chessboard views from the chessboard space to world space.
但是，我感兴趣的是全局外部矩阵，也就是说，一旦我移除了棋盘格，我希望能够在图像场景中指定一个点，即x，y及其高度，并给我指定位置世界空间.据我了解，我需要内在和外在的矩阵.人们应该如何从这里开始计算外在矩阵?我可以使用在棋盘校准步骤中已经收集的测量结果来计算外部矩阵吗?
However, what I am interested in is the global extrinsic matrix i.e. once I have removed the checkerboard, I want to be able to specify a point in the image scene i.e. x, y and its height and it gives me the position in the world space. As far as I understand, I need both the intrinsic and extrinsic matrix for this. How should one proceed to compute the extrinsic matrix from here? Can I use the measurements that I have already gathered from the chessboard calibration step to compute the extrinsic matrix as well?
推荐答案
让我放置一些上下文.考虑以下图片，(来自 https://docs.opencv.org /2.4/modules/calib3d/doc/camera_calibration_and_3d_reconstruction.html ):
Let me place some context. Consider the following picture, (from https://docs.opencv.org/2.4/modules/calib3d/doc/camera_calibration_and_3d_reconstruction.html):
相机已连接"刚性参照系(Xc，Yc，Zc).成功执行的固有校准可让您将点(Xc，Yc，Zc)转换为其在图像(u，v)上的投影以及在图像中的点(u，v)的投影到(Xc，Yc，Zc)中的光线(您只能将其放大至缩放比例).
The camera has "attached" a rigid reference frame (Xc,Yc,Zc). The intrinsic calibration that you successfully performed allows you to convert a point (Xc,Yc,Zc) into its projection on the image (u,v), and a point (u,v) in the image to a ray in (Xc,Yc,Zc) (you can only get it up to a scaling factor).
在实践中，您要将相机放置在外部世界"参考框架中，我们称其为(X，Y，Z).然后有一个刚性变换，由旋转矩阵 R 和平移矢量 T 表示，例如:
In practice, you want to place the camera in an external "world" reference frame, let's call it (X,Y,Z). Then there is a rigid transformation, represented by a rotation matrix, R, and a translation vector T, such that:
|Xc| |X|
|Yc|= R |Y| + T
|Zc| |Z|
这是外部校准(也可以写为4x4矩阵，也就是外部矩阵).
That's the extrinsic calibration (which can be written also as a 4x4 matrix, that's what you call the extrinsic matrix).
现在，答案.要获取 R 和 T ，您可以执行以下操作:
Now, the answer. To obtain R and T, you can do the following:
-
固定您的世界参考系，例如，地面可以是(x，y)平面，然后为其选择原点.
Fix your world reference frame, for example the ground can be the (x,y) plane, and choose an origin for it.
在此参考系中设置一些坐标已知的点，例如，地板上正方形网格中的点.
Set some points with known coordinates in this reference frame, for example, points in a square grid in the floor.
拍照并获取相应的2D图像坐标.
Take a picture and get the corresponding 2D image coordinates.
Use solvePnP to obtain the rotation and translation, with the following parameters:
- objectPoints:世界参考系中的3D点.
- imagePoints:图像中对应的2D点与objectPoints顺序相同.
- cameraMatris:您已经拥有的内在矩阵.
- distCoeffs:您已经拥有的失真系数.
- rvec ， tvec :这些将是输出.
- useExtrinsicGuess:错误
- 标志:您可以使用CV_ITERATIVE
- objectPoints: the 3D points in the world reference frame.
- imagePoints: the corresponding 2D points in the image in the same order as objectPoints.
- cameraMatris: the intrinsic matrix you already have.
- distCoeffs: the distortion coefficients you already have.
- rvec, tvec: these will be the outputs.
- useExtrinsicGuess: false
- flags: you can use CV_ITERATIVE
最后，使用
Finally, get R from rvec with the Rodrigues function.
您至少需要3个非共线点以及相应的3D-2D坐标才能使solvePnP工作(
You will need at least 3 non-collinear points with corresponding 3D-2D coordinates for solvePnP to work (link), but more is better. To have good quality points, you could print a big chessboard pattern, put it flat in the floor, and use it as a grid. What's important is that the pattern is not too small in the image (the larger, the more stable your calibration will be).
而且，非常重要:对于内在校准，您使用了具有一定大小正方形的象棋图案，但是您告诉算法(该算法对每种图案都执行solvePnPs)，每个方块的大小为 1 .这不是很明显，但是是在示例代码的第10行中完成的，其中使用坐标0,1,2，...:
And, very important: for the intrinsic calibration, you used a chess pattern with squares of a certain size, but you told the algorithm (which does kind of solvePnPs for each pattern), that the size of each square is 1. This is not explicit, but is done in line 10 of the sample code, where the grid is built with coordinates 0,1,2,...:
objp [:，:2] = np.mgrid [0:7,0:6] .T.reshape(-1,2)
objp[:,:2] = np.mgrid[0:7,0:6].T.reshape(-1,2)
外部校准的世界范围必须与此匹配，因此您有几种可能性:
And the scale of the world for the extrinsic calibration must match this, so you have several possibilities:
-
使用相同的比例，例如通过使用相同的网格或以相同的比例测量世界"平面的坐标.在这种情况下，您的世界"规模将不合适.
Use the same scale, for example by using the same grid or by measuring the coordinates of your "world" plane in the same scale. In this case, you "world" won't be at the right scale.
推荐:以正确的比例重做固有校准，例如:
Recommended: redo the intrinsic calibration with the right scale, something like:
objp [:，:2] =(size_of_a_square * np.mgrid [0:7,0:6]).T.reshape(-1,2)
objp[:,:2] = (size_of_a_square*np.mgrid[0:7,0:6]).T.reshape(-1,2)
其中size_of_a_square是正方形的实际大小.
Where size_of_a_square is the real size of a square.
(尚未执行此操作，但从理论上讲是可能的，如果您无法执行此操作，则执行此操作2)通过缩放fx和fy来重用固有校准.之所以可能这样做是因为相机可以看到所有比例尺，并且正方形的声明大小仅会更改fx和fy(每个正方形的姿势中的 T ，但这是另一回事了).如果正方形的实际大小为 L ，则在调用solvePnP之前，先替换fx和fy L fx和L fy.
(Haven't done this, but is theoretically possible, do it if you can't do 2) Reuse the intrinsic calibration by scaling fx and fy. This is possible because the camera sees everything up to a scale factor, and the declared size of a square only changes fx and fy (and the T in the pose for each square, but that's another story). If the actual size of a square is L, then replace fx and fy Lfx and Lfy before calling solvePnP.
这篇关于用opencv进行外在矩阵计算的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
