OpenCV-倾斜的摄像头和三角测量地标，可实现立体视觉 [英] OpenCV - Tilted camera and triangulation landmark for stereo vision

问题描述

我正在使用立体声系统，因此我试图通过三角测量获得某些点的世界坐标.

I am using a stereo system and so I am trying to get world coordinates of some points by triangulation.

我的相机呈现一个角度，Z轴方向(深度方向)不垂直于我的表面.这就是为什么当我观察平面时，我得到的不是恒定的深度而是线性"变化，对吗?我想要从基线方向开始的深度...如何重新投影?

My cameras present an angle, the Z axis direction (direction of the depth) is not normal to my surface. That is why when I observe flat surface, I get no constant depth but a "linear" variation, correct? And I want the depth from the baseline direction... How I can re-project?

我的一段带有投影数组和三角函数的代码:

A piece of my code with my projective arrays and triangulate function :

#C1 and C2 are the cameras matrix (left and rig)
#R_0 and T_0 are the transformation between cameras
#Coord1 and Coord2 are the correspondant coordinates of left and right respectively
P1 = np.dot(C1,np.hstack((np.identity(3),np.zeros((3,1))))) 

P2 =np.dot(C2,np.hstack(((R_0),T_0)))

for i in range(Coord1.shape[0])
    z = cv2.triangulatePoints(P1, P2, Coord1[i,],Coord2[i,])

--------稍后编辑-----------

-------- EDIT LATER -----------

感谢涂抹，所以我尝试应用您的建议.但是我认为我有一个错误，因为它无法正常运行，如下所示.而且点云似乎已弯曲并朝着图像的边缘弯曲.

Thanks scribbleink, so i tried to apply your proposal. But i think i have a mistake because it doesnt work well as you can see below. And the point clouds seems to be warped and curved towards the edges of the image.

U, S, Vt = linalg.svd(F)
V = Vt.T

#Right epipol
U[:,2]/U[2,2]

# The expected X-direction with C1 camera matri and C1[0,0] the focal length
vecteurX = np.array([(U[:,2]/U[2,2])[0],(U[:,2]/U[2,2])[1],C1[0,0]])
vecteurX_unit = vecteurX/np.sqrt(vecteurX[0]**2 + vecteurX[1]**2 + vecteurX[2]**2)


# The expected Y axis :
height = 2048
vecteurY = np.array([0, height -1, 0])
vecteurY_unit = vecteurY/np.sqrt(vecteurY[0]**2 + vecteurY[1]**2 + vecteurY[2]**2)


# The expected Z direction :
vecteurZ = np.cross(vecteurX,vecteurY)
vecteurZ_unit = vecteurZ/np.sqrt(vecteurZ[0]**2 + vecteurZ[1]**2 + vecteurZ[2]**2)

#Normal of the Z optical (the current Z direction)
Zopitcal = np.array([0,0,1])

cos_theta = np.arccos(np.dot(vecteurZ_unit, Zopitcal)/np.sqrt(vecteurZ_unit[0]**2 + vecteurZ_unit[1]**2 + vecteurZ_unit[2]**2)*np.sqrt(Zopitcal[0]**2 + Zopitcal[1]**2 + Zopitcal[2]**2))

sin_theta = (np.cross(vecteurZ_unit, Zopitcal))[1]

#Definition of the Rodrigues vector and use of cv2.Rodrigues to get rotation matrix
v1 = Zopitcal  
v2 = vecteurZ_unit 

v_rodrigues = v1*cos_theta + (np.cross(v2,v1))*sin_theta + v2*(np.cross(v2,v1))*(1. - cos_theta)
R = cv2.Rodrigues(v_rodrigues)[0]

推荐答案

要添加到 fireant 的响应中，请在此处是一个候选解决方案，假设预期的X方向与连接两个摄像机的投影中心的线重合.

To add to fireant's response, here is one candidate solution, assuming that the expected X-direction coincides with the line joining the centers of projection of the two cameras.

1. 通过针孔模型校准计算焦距f_1和f_2.
2. 解决摄像机2的极点在摄像机1的框架中的位置.为此，您可以使用立体摄像机对的基本矩阵(F)或基本矩阵(E).具体来说，左右两极位于F的零空间中，因此您可以使用奇异值分解.有关扎实的理论参考，请参见第246页上的Hartley和Zisserman，第二版，表9.1基本矩阵特性的摘要"(其他stackoverflow帖子.

1. Compute the focal lengths f_1 and f_2 (via pinhole model calibration).
2. Solve for the location of camera 2's epipole in camera 1's frame. For this, you can use either the Fundamental matrix (F) or the Essential matrix (E) of the stereo camera pair. Specifically, the left and right epipoles lie in the nullspace of F, so you can use Singular Value Decomposition. For a solid theoretical reference, see Hartley and Zisserman, Second edition, Table 9.1 "Summary of fundamental matrix properties" on Page 246 (freely available PDF of the chapter).
3. The center of projection of camera 1, i.e. (0, 0, 0) and the location of the right epipole, i.e. (e_x, e_y, f_1) together define a ray that aligns with the line joining the camera centers. This can be used as the expected X-direction. Call this vector v_x.
4. Assuming that the expected Y axis faces downward in the image plane, i.e, from (0, 0, f_1) to (0, height-1, f_1), where f is the focal length. Call this vector as v_y.
5. The expected Z direction is now the cross-product of vectors v_x and v_y.
6. Using the expected Z direction along with the optical axis (Z-axis) of camera 1, you can then compute a rotation matrix from two 3D vectors using, say the method listed in this other stackoverflow post.

实用说明: 根据我的实际经验，如果没有付出很大的努力，就不可能期望平面对象与立体声基线完全对齐.需要一定数量的飞机装配和额外的旋转.

Practical note: Expecting the planar object to exactly align with the stereo baseline is unlikely without considerable effort, in my practical experience. Some amount of plane-fitting and additional rotation would be required.

一次性工作: 这取决于您是否需要执行一次，例如对于一次校准，在这种情况下，只需实时进行此估计过程，然后旋转立体声摄像机对，直到将深度图差异最小化即可.然后锁定您的相机位置，并祈祷以后不要有人撞到它.

One-time effort: It depends on whether you need to do this once, e.g. for one-time calibration, in which case simply make this estimation process real-time, then rotate your stereo camera pair until the depth map variance is minimized. Then lock your camera positions and pray someone doesn't bump into it later.

可重复性: 如果您需要继续将估计的深度图与真正随心所欲的Z轴对齐，这些Z轴会随捕获的每个新帧而变化，那么您应该考虑在平面估算方法上花费时间并使其更加可靠.

Repeatability: If you need to keep aligning your estimated depth maps to truly arbitrary Z-axes that change for every new frame captured, then you should consider investing time in the plane-estimation method and making it more robust.

这篇关于OpenCV-倾斜的摄像头和三角测量地标，可实现立体视觉的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！