python中的3D图像旋转 [英] 3D image rotation in python

问题描述

我有以下图像 I1.我没有拍到.我是从谷歌下载的

我将已知的单应性 h 应用于 I1 以获得以下图像 I2.

我想假设相机已经拍摄了上面这张 I2 的照片.我找到了这个相机"的相机矩阵.让这个相机矩阵为 k.现在，我想围绕相机轴旋转此图像 I2.根据

0.00006

0.00009

为什么旋转仅适用于如此小的 theta_in_degrees 值?此外，图像中可见的旋转实际上并不是围绕 z 轴发生的.为什么图像不绕 z 轴旋转?我哪里出错了，我该如何解决这些问题?

h 矩阵:

[[ 1.71025842e+00 -7.51761942e-01 1.02803446e+02][ -2.98552735e-16 1.39232576e-01 1.62792482e+02][ -1.13518150e-18 -2.27094753e-03 1.00000000e+00]]

k 矩阵:

[[ 1.41009391e+09 0.00000000e+00 5.14000000e+02][ 0.00000000e+00 1.78412347e+02 1.17000000e+02][ 0.00000000e+00 0.00000000e+00 1.00000000e+00]]

在采纳 Toby Collins 的建议后，我将 k 的左上角值设置为与 k[1][1] 相同.当我现在围绕 z 轴执行旋转时，我获得了 theta_in_degrees 的所有值的正确旋转图像，从 0 到 360.但是，当我尝试通过更改围绕 y 轴旋转图像时将上面代码中的ux、uy和uz改为以下，我得到了荒谬的旋转结果:

ux=0.0你=1.0乌兹=0.0

theta_in_degrees 不同值的一些示例以及绕 y 轴旋转的相应结果如下所示:

-10

-40

-90

-110

我哪里还出错了?另外，为什么旋转图像中连续黄色条纹的长度和宽度会出现如此巨大的下降?以及为什么图像的一部分会环绕(例如，旋转 -90 和 -110 度的结果)?

我的问题的第二部分是:我的旋转轴的向量方程是 (320, 0, -10)+t(0, 1, 0).为了使用这个方法，计算旋转矩阵，我需要定义ux、uy 和 uz 的旋转轴使得 ux^2+uy^2+uz^2=1.如果需要围绕坐标轴之一进行旋转(正如我目前为测试目的所做的那样)，这将很简单.但是如果 t 在我的向量方程中，我如何获得 ux、uy 和 uz 的这些值旋转轴是可变的?我也愿意接受有关寻找合适的旋转矩阵 R 的任何其他方法的建议，以便旋转围绕我提到的轴进行(例如，x 度).

解决方案

您遇到的困难是您的单应矩阵 h 与使用 sensible<获得的投影不一致/em> 透视相机.我认为有一个更简单的方法.

从根本上说，您需要非常清楚自己的技术目标，并将其与解决它的方法分开.每当您解决任何视力问题时，请始终这样做.

技术目标

所以让我们明确一下技术目标.您有一个平面的自顶向下图像(也称为修正视图).通常你会称这个表面为 model，定义在平面 z=0 上.你想渲染这个模型.具体你要做到以下几点；

1. 创建了一个从特定视点观察模型的虚拟透视相机.我们通过 R1、t1 和一个内在矩阵 K 来定义模型到相机的刚性变换.
2. 相机通过围绕其投影中心旋转来移动.让我们用 R2 表示这个旋转.
3. 模型是使用 2 中的视图渲染的.我们将此图像称为 I2

为简单起见，我将使用 T(R,t) 来表示某些旋转的 4x4 均匀刚性变换 R 和翻译t.因此，第 3 阶段的模型到相机变换为 T=T(R2, (0,0,0)) xT(R1, t1).

渲染选项

有两种创建I2

的好方法

1. 使用渲染引擎，例如 OpenGL 或 Ogre.这样做的好处是可以很容易地制作用于更改相机视点的 GUI，并且可以添加其他复杂的渲染效果.

2. 确定模型到图像的单应矩阵，并使用 warpPerspective 使用 OpenCV 进行渲染.这样做的好处是可以在几行中完成，而无需中断渲染软件.缺点是，如果单应性在渲染中有消失点(正如您所观察到的)，您可能会得到一些奇怪的效果.稍后会详细介绍这一点.

模型到图像单应的定义

为了使用 OpenCV 方法，我们将 model-to-image 单应定义为 H2.这可以根据相机参数来定义.考虑齐次坐标模型平面上的一个点 p=(x,y,1).它在 I2 中齐次坐标中的位置 q 由 q=K M p 给出，其中 M 是.M=(T00,T01,T03; T10,T11,T13; T20,T21,T23) 给出的 3x3 矩阵.这很容易使用透视相机模型推导出来.因此，我们现在有 H2 =K M.

实例化单应矩阵

现在我们必须实例化单应性，与您提出的方法不同，我将使用特定的相机配置来定义它，方法是指定 K、R1、t1，R2.这个选择由你！为了简化K的定义，您可以使用一个带有一个自由参数(焦距)的简单形式，并将主点设置为图像中心.对于典型的相机，f 的范围在图像宽度的 0.5 到 2 倍之间，但这取决于您.然后，您需要根据您的视点所需的视角/距离设置 R1 和 t1.

这与您目前的方法有何不同

我想强调的是，这与我之前给出的任何答案都不矛盾.这只是一种不同的方法，可能更易于管理.从本质上讲，在这里我建议直接使用相机参数(您根据需要设置)来定义您的单应性.这保证您使用的是合理的内在矩阵(因为您自己设置了它).这与您首先创建单应性然后想要找到匹配的相机参数(这可能是也可能不是物理上合理的)的方法不同.

I have the following image I1. I did not capture it. I downloaded it from Google

I apply a known homography h to I1 to obtain the following image I2.

I want to assume that a camera has taken this above shot of I2. I have found the camera matrix of this "camera". Let this camera matrix be k. Now, I want to rotate this image I2 about the camera axis. According to the explanation in the accepted answer in this question, I need to set the rotation matrix R and then perform k*R*inv(k)*h on image I1 to get the required rotated image I3.

I have been facing problems when I try to set this rotation matrix R. I have used this method to set the matrix R.

To test my code, I initially tried to rotate the image around the z-axis by 10 degrees but I wasn't getting the correct output.

My partial Python code:

theta_in_degrees = 10
theta_in_radians = theta_in_degrees*math.pi/180
ux=0.0 
uy=0.0 
uz=1.0 
vector_normalize_factor = math.sqrt(ux*ux+uy*uy+uz*uz)
ux=ux/vector_normalize_factor
uy=uy/vector_normalize_factor
uz=uz/vector_normalize_factor
print "ux*ux+uy*uy+uz*uz = ", ux*ux+uy*uy+uz*uz 
rotation_matrix = np.zeros([3,3])
c1 = math.cos(theta_in_radians)
c2 = 1-c1
s1 = math.sin(theta_in_radians)
rotation_matrix[0][0] = c1+ux*ux*c2
rotation_matrix[0][1] = ux*uy*c2-uz*s1
rotation_matrix[0][2] = ux*uz*c2+uy*s1
rotation_matrix[1][0] = uy*ux*c2+uz*s1
rotation_matrix[1][1] = c1+uy*uy*c2
rotation_matrix[1][2] = uy*uz*c2-ux*s1
rotation_matrix[2][0] = uz*ux*c2-uy*s1
rotation_matrix[2][1] = uz*uy*c2+ux*s1
rotation_matrix[2][2] = c1+uz*uz*c2
print "rotation_matrix = ", rotation_matrix
R = rotation_matrix
#Calculate homography H1 between reference top view and rotated frame
k_inv = np.linalg.inv(k)
Hi = k.dot(R)
Hii = k_inv.dot(h)
H1 = Hi.dot(Hii)
print "H1 = ", H1
im_out = cv2.warpPerspective(im_src, H1, (im_dst.shape[1],im_dst.shape[0]))

Here, img_src is the source of I1.

The result I got when I tried the above code is a black image with no part of the image visible. However, when I changed the value of theta_in_degrees to the following values, these were my outputs:

0.00003

0.00006

0.00009

Why is the rotation working only for such small values of theta_in_degrees? Also, the rotation visible in the images is not actually happening around the z-axis. Why isn't the image rotating about the z-axis? Where am I going wrong and how can I fix these issues?

h matrix:

[[  1.71025842e+00  -7.51761942e-01   1.02803446e+02]
 [ -2.98552735e-16   1.39232576e-01   1.62792482e+02]
 [ -1.13518150e-18  -2.27094753e-03   1.00000000e+00]]

k matrix:

[[  1.41009391e+09   0.00000000e+00   5.14000000e+02]
 [  0.00000000e+00   1.78412347e+02   1.17000000e+02]
 [  0.00000000e+00   0.00000000e+00   1.00000000e+00]]

Edit:

After incorporating the suggestion by Toby Collins, I set the top left value of k to be the same as k[1][1]. When I now perform rotation about the z-axis, I get the correct rotated images for all values of theta_in_degrees from 0 to 360. However, when I try to rotate the image about the y-axis by changing the ux, uy and uz in the above code to the following, I get absurd rotation results:

ux=0.0 
uy=1.0 
uz=0.0 

Some samples for different values of theta_in_degrees and the corresponding results for rotation about the y-axis are shown below:

-10

-40

-90

-110

Where am I still going wrong? Also, why is there such a huge drop in the length and width of successive yellow stripes in a rotated image? And why does a part of the image wrap around (for example, the results of rotation by -90 and -110 degrees)?

The second part of my question is this: The vector equation of my axis of rotation is (320, 0, -10)+t(0, 1, 0). In order to use this method, to calculate the rotation matrix, I need to define the ux, uy and uz of the axis of rotation such that ux^2+uy^2+uz^2=1. This would be straightforward if the rotation needs to be done around one of the coordinate axes (as I am currently doing for testing purposes). But how do I get these values of ux, uy and uz if the t in the vector equation of my rotation axis is variable? I am also open to suggestions regarding any other approaches to finding a suitable rotation matrix R such that the rotation happens around the axis I have mentioned (say, by x degrees).

解决方案

The difficulty you are having is that your homography matrix h does not correspond well with a projection obtained with a sensible perspective camera. I think there is a simpler approach.

Fundamentally, you needed to be very clear about your technical goal, and separate this from your approach for solving it. Always do this whenever you tackle any vision problem.

Technical goal

So let's be clear about the technical goal. You have a top-down image of a planar surface (also called a rectified view). Normally you would call this surface the model, defined on the plane z=0. You want to render this model. Specifically you want to do the following;

1. A virtual perspective camera is created that is looking at the model from a particular viewpoint. We define this model-to-camera rigid transform by R1, t1, with an intrinsic matrix K.
2. The camera is moved by rotating it about its centre of projection. Let's denote this rotation by R2.
3. The model is rendered using the view from 2. We will call this image I2

For simplicity I'm going to use T(R,t) to denote the 4x4 homogeneous rigid transform for some rotation R and translation t. The model-to-camera transform at stage 3 is therefore given by T=T(R2, (0,0,0)) x T(R1, t1).

Rendering options

There are two good ways to create I2

1. Use a rendering engine such as OpenGL or Ogre. The advantage of this is that it can be easy to make a GUI for changing the camera viewpoint and other complex rendering effects can be added.

2. Determine the model-to-image homography matrix and render with OpenCV using warpPerspective. The advantage of this is that it can be done in a few lines without breaking into rendering software. The disadvantage is that you can get some strange effects if the homography has a vanishing point in the render (as you are observing). More on that point later.

Definition of model-to-image homography

To use the OpenCV approach we define the model-to-image homography as H2. This can be defined in terms of the camera parameters. Consider a point p=(x,y,1) on the model plane in homogeneous coordinates. Its position q in I2 in homogeneous coordinates is given by q=K M p, where M is. 3x3 matrix given by M=(T00,T01,T03; T10,T11,T13; T20,T21,T23). This is straightforward to derive using the perspective camera model. Consequently, we now have that H2 =K M.

Instantiate the homography matrix

Now we have to instantiate the homography, unlike your proposed approach, I would define it using a particular camera configuration, by specifying K, R1, t1, R2. The choice is up to you! To simplify the definition of K you can use a simple form with one free parameter (focal length), and set the principal point to the image centre. For typical cameras f ranges between 0.5 and 2 time the image width, but it's up to you. You then need to set R1 and t1 depending on the viewing angle/distance that you want for your viewpoint.

How is this different to your current approach

I want to emphasize that this does not contradict any of the previous answers I have given. It is simply a different approach which may be easier to manage. Essentially, here I am proposing to define your homography directly using camera parameters (which you set as you want). This guarantees you are using a sensible intrinsic matrix (because you set it yourself). It is different to your approach where you first create a homography and then want to find the matching camera parameters (which may or not be physically sensible).

这篇关于python中的3D图像旋转的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！