如何使用投影矩阵从图像计算真实坐标系中的射线? [英] How to calculate ray in real-world coordinate system from image using projection matrix?
问题描述
给定n幅图像和每幅图像的投影矩阵，我如何计算与现实坐标系统的三个平面之一相交的图像的每个像素发出的光线(线)?相机捕获的对象在同一位置，只是每张图像的相机位置都不同.这就是每个图像都有单独的投影矩阵的原因.
Given n images and a projection matrix for each image, how can i calculate the ray (line) emitted by each pixel of the images, which is intersecting one of the three planes of the real-world coordinate system? The object captured by the camera is at the same position, just the camera's position is different for each image. That's why there is a separate projection matrix for each image.
据我的研究表明，这是3D到2D投影的逆过程.由于投影到2D时信息会丢失，因此只能在现实世界的坐标系中计算射线(线)，这很好.
As far as my research suggests, this is the inverse of the 3D to 2D projection. Since information is lost when projecting to 2D, it's only possible to calculate the ray (line) in the real-world coordinate system, which is fine.
示例投影矩阵P，根据K * [R t]，根据给定的K，R和t分量计算得出
An example projection matrix P, that a calculated based on given K, R and t component, according to K*[R t]
3310.400000 0.000000 316.730000
K= 0.000000 3325.500000 200.550000
0.000000 0.000000 1.000000
-0.14396457836077139000 0.96965263281337499000 0.19760617153779569000
R= -0.90366580603479685000 -0.04743335255026152200 -0.42560419233334673000
-0.40331536459778505000 -0.23984130575212276000 0.88306936201487163000
-0.010415508744
t= -0.0294278883669
0.673097816109
-604.322 3133.973 933.850 178.711
P= -3086.026 -205.840 -1238.247 37.127
-0.403 -0.240 0.883 0.673
我正在使用 http://vision.middlebury.edu/上的"DinoSparseRing"数据集.mview/数据
for (int i = 0; i < 16; i++) {
RealMatrix rotationMatrix = MatrixUtils.createRealMatrix(rotationMatrices[i]);
RealVector translationVector = MatrixUtils.createRealVector(translationMatrices[i]);
// construct projection matrix according to K*[R t]
RealMatrix projMatrix = getP(kalibrationMatrices[i], rotationMatrices[i], translationMatrices[i]);
// getM returns the first 3x3 block of the 3x4 projection matrix
RealMatrix projMInverse = MatrixUtils.inverse(getM(projMatrix));
// compute camera center
RealVector c = rotationMatrix.transpose().scalarMultiply(-1.f).operate(translationVector);
// compute all unprojected points and direction vector per project point
for (int m = 0; m < image_m_num_pixel; m++) {
for (int n = 0; n < image_n_num_pixel; n++) {
double[] projectedPoint = new double[]{
n,
m,
1};
// undo perspective divide
projectedPoint[0] *= projectedPoint[2];
projectedPoint[1] *= projectedPoint[2];
// undo projection by multiplying by inverse:
RealVector projectedPointVector = MatrixUtils.createRealVector(projectedPoint);
RealVector unprojectedPointVector = projMInverse.operate(projectedPointVector);
// compute direction vector
RealVector directionVector = unprojectedPointVector.subtract(c);
// normalize direction vector
double dist = Math.sqrt((directionVector.getEntry(0) * directionVector.getEntry(0))
+ (directionVector.getEntry(1) * directionVector.getEntry(1))
+ (directionVector.getEntry(2) * directionVector.getEntry(2)));
directionVector.setEntry(0, directionVector.getEntry(0) * (1.0 / dist));
directionVector.setEntry(1, directionVector.getEntry(1) * (1.0 / dist));
directionVector.setEntry(2, directionVector.getEntry(2) * (1.0 / dist));
}
}
}
以下2个图显示了每个图像的外部光线(总共16张图像).蓝色端是相机点，青色是包含相机捕获的对象的边界框.人们可以清楚地看到射线在世界坐标系中投射回对象.
The following 2 plots show the outer rays for each images (total of 16 images). The blue end is the camera point and the cyan is a bounding box containing the object captured by the camera. One can clearly see the rays projecting back to the object in world coordinate system.
推荐答案
要定义光线，您需要一个起点(即相机/眼睛位置)和一个方向向量，可以使用光线上的任何点来计算
To define the ray you need a start point (which is the camera/eye position) and a direction vector, which can be calculated using any point on the ray.
对于图像中的给定像素，您有一个投影的X和Y(在图像中心置零)，但没有Z深度值.但是，与该像素所有可能的深度值相对应的真实坐标将全部位于您要计算的射线上，因此您可以选择任意任意非零Z值，因为射线上的任何点都可以
For a given pixel in the image, you have a projected X and Y (zeroed at the center of the image) but no Z depth value. However the real-world co-ordinates corresponding to all possible depth values for that pixel will all lie on the ray you are trying to calculate, so you can just choose any arbitrary non-zero Z value, since any point on the ray will do.
float projectedX = (x - imageCenterX) / (imageWidth * 0.5f);
float projectedY = (y - imageCenterY) / (imageHeight * 0.5f);
float projectedZ = 1.0f; // any arbitrary value
现在有了3D投影坐标，您可以通过以下方式撤消投影:将X和Y乘以Z，然后将结果乘以反投影矩阵，以得到未投影的点.>
Now that you have a 3D projected co-ordinate you can undo the projection by applying the perspective divide in reverse by multiplying X and Y by Z, then multiplying the result by the inverse projection matrix to get the unprojected point.
// undo perspective divide (redundant if projectedZ = 1, shown for completeness)
projectedX *= projectedZ;
projectedY *= projectedZ;
Vector3 projectedPoint = new Vector3(projectedX, projectedY, projectedZ);
// undo projection by multiplying by inverse:
Matrix invProjectionMat = projectionMat.inverse();
Vector3 unprojectedPoint = invProjectionMat.multiply(projectedPoint);
从非投影点减去摄影机位置，以获取从摄影机到该点的方向向量，然后对其进行归一化.(此步骤假定投影矩阵同时定义了相机的位置和方向，如果位置是单独存储的，则无需进行减法运算)
Subtract the camera position from the unprojected point to get the direction vector from the camera to the point, and then normalize it. (This step assumes that the projection matrix defines both the camera position and orientation, if the position is stored separately then you don't need to do the subtraction)
Vector3 directionVector = unprojectedPoint.subtract(cameraPosition);
directionVector.normalize();
射线由相机位置和归一化方向矢量定义.然后，您可以将其与X，Y，Z平面中的任何一个相交.
The ray is defined by the camera position and the normalized direction vector. You can then intersect it with any of the X, Y, Z planes.
这篇关于如何使用投影矩阵从图像计算真实坐标系中的射线?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
