OpenCV:使用solvePnP确定单应性 [英] OpenCV: use solvePnP to determine homography
问题描述
在过去的几周里，我尝试学习纠正图像，并且在这里的人们的帮助下，我已经更好地理解了图像.大约一周前，我建立了一个要纠正的测试示例(从上方查看图像).效果很好(原始: http://sitedezign.net/original.jpg 并进行了纠正: http://sitedezign.net/rectified.jpg )具有函数T = cv2.getPerspectiveTransform(UV_cp，XYZ_gcp)，其中T成为同形文字.
Over the past weeks I've tried learning to rectify images and with the help of the people here I've managed to understand it better. About a week ago I set up a test example which I wanted to rectify (view the image from above). This worked fine (original: http://sitedezign.net/original.jpg and rectified: http://sitedezign.net/rectified.jpg) with the function T = cv2.getPerspectiveTransform(UV_cp, XYZ_gcp) where T becomes the Homography.
当我尝试使用真实世界的照片进行此操作时，它失败了，因为真实世界的坐标并非完美地在平面上(而是在空间中的X，Y和Z坐标中仅测量了约10个控制点).因此，我决定使用solvePnP，并希望能够创建一个我可以使用的同形文字.
When I tried to do this with real world photos it failed because the realworld coordinates aren't perfectly on a plane (but simply around 10 control points measured in X, Y and Z coordinates in space). Therefor I decided to use solvePnP and hopefully be able to create a Homography which I can use.
我在测试示例上尝试了此操作，但是没有得到我期望的结果:图像没有得到校正，并且我使用SolvePnP计算的Homography不等于使用getPerspectiveTransform计算的Homography.
I tried this on the test example but didn't get the results I expected: the image isn't rectified, and my Homography calculated using solvePnP isn't equal to the Homography calculated with getPerspectiveTransform.
我的代码:
# Set UV (image) and XYZ (real life)
UV_cp = np.array([[1300.0, 2544.0], # left down
[1607.0, 1000.0], # left up
[3681.0, 2516.0], # right down
[3320.0, 983.0]], np.float32) # right up
# Z is on 0 plane, so Z=0.0
XYZ_gcp = np.array([[0.0, 400.0, 0.0],
[0.0, 0.0, 0.0],
[300.0, 400.0, 0.0],
[300.0, 0.0, 0.0]], np.float32)
rvec, tvec = cv2.solvePnP(XYZ_gcp, UV_cp, K, D)
rotM_cam = cv2.Rodrigues(rvec)[0]
# calculate camera position (= translation), in mm from 0,0,0 point
cameraPosition = -np.matrix(rotM_cam).T * np.matrix(tvec)
# 3x3 Identity matrix
I = np.identity(3)
# [I|-C]
I1_extended = np.hstack((I,-cameraPosition))
# P = K*R*I
P_cam = K.dot(rotM_cam).dot(I1_extended)
# create P2 = image from above: R = 0,0,0, translation = x, y, z = 0,0,-1000 (mm)
R_rec = matr.getR(0.0,0.0,0.0)
newZ = -1000.0
new_cameraPosition = np.array([[0.0],[0.0],[newZ]])
I2_extended = np.hstack((I,new_cameraPosition))
P_rec = K.dot(R_rec).dot(I2_extended)
# correct Homography T from getPerspectiveTransform:
T = np.array([[4.70332834e-01, 9.35182514e-02, -4.24671558e+02],
[9.62104844e-03, 9.69462117e-01, -4.92461571e+02],
[3.54859924e-06, 6.80081146e-04, 1.00000000e+00]])
# Homography Matrix = H = P_rect * pinv(P) => P2 * pinv(P1)
H = P_rec.dot(np.linalg.pinv(P_cam))
结果是扭曲的图像，远不等于上面显示的图像(校正后的图像).同样，应正确的单应性T(来自getPerspectiveTransform)与使用SolvePnP(H)的结果计算出的单应性也不接近.
The result is a warped image, which is far from equal to the one shown above (the rectified one). Also the Homography T which should be correct (from getPerspectiveTransform) is not close to equal to the homography calculated using the result from solvePnP (H).
H from solvePnP:
[[ 1.01865631e+00 2.68683332e-01 -2.04519580e+03]
[ -3.24304366e-02 6.82672680e-01 -1.15688010e+03]
[ 2.03399902e-05 1.24191993e-04 -5.41378561e-01]]
H from getPerspectiveTransform:
[[ 4.70332834e-01 9.35182514e-02 -4.24671558e+02]
[ 9.62104844e-03 9.69462117e-01 -4.92461571e+02]
[ 3.54859924e-06 6.80081146e-04 1.00000000e+00]]
有人知道出了什么问题吗?
Anyone has an idea what is going wrong?
PS:用于确定K矩阵和失真系数的代码(值是根据Adobe Camera Raw从33mm焦距的我的相机Pentax K-5上获取的):
PS: The code to determine the K matrix and distortion coefficients (values are taken from my camera Pentax K-5 at 33mm focal length according to Adobe Camera Raw):
# Focal length, sensor size (mm and px)
f = 33.0 # mm
pix_width = 4928.0 # sensor size has 4928px in width
pix_height = 3624.0 # sensor size has 4928px in width
sensor_width = 23.7 # mm
sensor_height = 15.7 # mm
# set center pixel
u0 = int(pix_width / 2.0)
v0 = int(pix_height / 2.0)
# determine values of camera-matrix
mu = pix_width / sensor_width # px/mm
alpha_u = f * mu # px
mv = pix_height / sensor_height # px/mm
alpha_v = f * mv # px
# Distortion coefs
D = np.array([[0.0, 0.0, 0.0, 0.0]])
# Camera matrix
K = np.array([[alpha_u, 0.0, u0],
[0.0, alpha_v, v0],
[0.0, 0.0, 1.0]])
推荐答案
您的K矩阵似乎合适，但这可能不足以实现真实图像的良好准确性.我认为您应该使用calibrateCamera函数(
Your K matrix seems appropriate, but this might not be sufficient to achieve good accuracy with real images. I think instead of giving a priori reasonable values (in particular for the optical center pixel & lens distortion coefficients), you should calibrate your camera using the calibrateCamera function (documentation link, tutorials). However, I don't think the problem you are mentionning is caused by that.
我认为您的问题来自P_rec 的定义.
I think your problem comes from the definition of P_rec.
首先，请注意，如果您使用newZ = -1000.0，则实际上是在将相机平移1000米(不是毫米).
First, be aware that if you use newZ = -1000.0, you are actually translating the camera by 1000 meters (not millimeters).
第二，您必须非常小心要考虑的3D点以及要将它们投影到图像中的位置:
Second, you have to be very careful what 3D points you are considering and where you want them to be projected in the image:
-
就像在
solvePnP函数中使用XYZ_gcp一样，这意味着您将这些坐标用作3D点.
As you used
XYZ_gcpin thesolvePnPfunction, this means that you are using these coordinates as 3D points.
在getPerspectiveTransform函数中使用XYZ_gcp时，这意味着您还将这些坐标用作2D坐标.请注意，严格来说，您不能执行此操作，因为getPerspectiveTransform需要两个4x2数组(而不是4x2和4x3)，但是我假设您删除了始终为0的第三个坐标.
As you used XYZ_gcp in the getPerspectiveTransform function, this means that you are also using these as 2D coordinates. Notice that, strictly speaking, you cannot do this because getPerspectiveTransform expects two 4x2 arrays (not a 4x2 and a 4x3), but I'll assume you dropped the third coordinates which are always 0.
因此，您的P_rec应该定义为[x; y; 1] = P_rec * [x; y; 0; 1].因此，P_rec应该定义如下:
Hence, your P_rec should be defined so that [x; y; 1] = P_rec * [x; y; 0; 1]. Therefore, P_rec should be defined as follows:
P_rec = [ [1 0 0 0] [0 1 0 0] [0 0 0 1] ].
这篇关于OpenCV:使用solvePnP确定单应性的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
