相机位置在世界坐标从cv :: solvePnP [英] Camera position in world coordinate from cv::solvePnP

问题描述

我有一个校准的相机（固有矩阵和失真系数），我想知道相机的位置知道一些3d点和它们的图像中的对应点（2d点）。

I have a calibrated camera (intrinsic matrix and distortion coefficients) and I want to know the camera position knowing some 3d points and their corresponding points in the image (2d points).

我知道 cv :: solvePnP 可以帮助我，在阅读这和这个我知道solvePnP rvec 和 tvec 的输出是旋转

I know that cv::solvePnP could help me, and after reading this and this I understand that I the outputs of solvePnP rvec and tvec are the rotation and translation of the object in camera coordinate system.

因此，我需要在世界坐标系中找出相机的旋转/平移。

So I need to find out the camera rotation/translation in the world coordinate system.

从上面的链接，似乎代码很简单，在python：

From the links above it seems that the code is straightforward, in python:

found,rvec,tvec = cv2.solvePnP(object_3d_points, object_2d_points, camera_matrix, dist_coefs)
rotM = cv2.Rodrigues(rvec)[0]
cameraPosition = -np.matrix(rotM).T * np.matrix(tvec)

我不知道python / numpy东西（我使用C ++）对我很有意义：

I don't know python/numpy stuffs (I'm using C++) but this does not make a lot of sense to me:

• rvec，solvePnP的tvec输出是3x1矩阵，3元素向量


• cv2.Rodrigues（rvec）是一个3x3矩阵


• cv2.Rodrigues（rvec）[0]是一个3x1矩阵，3个元素向量


• cameraPosition是一个3x1 * 1x3矩阵乘法，它是一个3x3矩阵。我如何在opengl中用简单的 glTranslatef 和 glRotate 调用？




• rvec, tvec output from solvePnP are 3x1 matrix, 3 element vectors
• cv2.Rodrigues(rvec) is a 3x3 matrix
• cv2.Rodrigues(rvec)[0] is a 3x1 matrix, 3 element vectors
• cameraPosition is a 3x1 * 1x3 matrix multiplication that is a.. 3x3 matrix. how can I use this in opengl with simple glTranslatef and glRotate calls?

推荐答案

如果世界坐标的意思是对象坐标，你必须得到由pnp给出的结果的逆变换算法。

If with "world coordinates" you mean "object coordinates", you have to get the inverse transformation of the result given by the pnp algorithm.

有一个反转换矩阵的技巧，它允许你保存反演操作，这通常是昂贵的，并且解释了Python中的代码。给定转换 [R | t] ，我们有 inv（[R | t]）= [R'| -R'* t] ，其中 R'是 R 的转置。所以，你可以编码（未测试）：

There is a trick to invert transformation matrices that allows you to save the inversion operation, which is usually expensive, and that explains the code in Python. Given a transformation [R|t], we have that inv([R|t]) = [R'|-R'*t], where R' is the transpose of R. So, you can code (not tested):

cv::Mat rvec, tvec;
solvePnP(..., rvec, tvec, ...);
// rvec is 3x1, tvec is 3x1

cv::Mat R;
cv::Rodrigues(rvec, R); // R is 3x3

R = R.t();  // rotation of inverse
tvec = -R * tvec; // translation of inverse

cv::Mat T(4, 4, R.type()); // T is 4x4
T( cv::Range(0,3), cv::Range(0,3) ) = R * 1; // copies R into T
T( cv::Range(0,3), cv::Range(3,4) ) = tvec * 1; // copies tvec into T
// fill the last row of T (NOTE: depending on your types, use float or double)
double *p = T.ptr<double>(3);
p[0] = p[1] = p[2] = 0; p[3] = 1;

// T is a 4x4 matrix with the pose of the camera in the object frame


$ b b

更新：稍后，要使用 T 与OpenGL，你必须记住，摄像机帧的轴在OpenCV和OpenGL。

Update: Later, to use T with OpenGL you have to keep in mind that the axes of the camera frame differ between OpenCV and OpenGL.

OpenCV使用计算机视觉中通常使用的引用：X指向右边，Y向下，Z指向前边（如此图片）。 OpenGL中的摄像机的框架是：X指向右侧，Y向上，Z指向后（如此图片）。因此，您需要围绕X轴旋转180度。此旋转矩阵的公式位于维基百科中。

OpenCV uses the reference usually used in computer vision: X points to the right, Y down, Z to the front (as in this image). The frame of the camera in OpenGL is: X points to the right, Y up, Z to the back (as in this image). So, you need to apply a rotation around X axis of 180 degrees. The formula of this rotation matrix is in wikipedia.

// T is your 4x4 matrix in the OpenCV frame
cv::Mat RotX = ...; // 4x4 matrix with a 180 deg rotation around X
cv::Mat Tgl = T * RotX; // OpenGL camera in the object frame

这些转换总是令人困惑， ，因此要用一粒盐。

These transformations are always confusing and I may be wrong at some step, so take this with a grain of salt.

最后，考虑到OpenCV中的矩阵以内存中的row-major顺序存储，OpenGL中的矩阵 - 大副。

Finally, take into account that matrices in OpenCV are stored in row-major order in memory, and OpenGL ones, in column-major order.

这篇关于相机位置在世界坐标从cv :: solvePnP的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！