在python opengl中使用图像中的2d点获取空间中的3d点 [英] get 3d point in space using 2d point in image in python opengl
问题描述
我正在尝试在房间中模拟深度相机，我的相机能够在世界中移动和旋转，并且房间被模拟为围绕 (0,0,0) 的 3d 立方体单击一个按钮，我想对图像中的 N 个随机点进行采样，并获得这些点与相机的距离(现实世界"中的距离).到目前为止，我已经成功地创建了移动相机和立方体的场景(示例)
Im trying to simulate a depth camera in a room, my camera is able to move and rotate in the world and the room is simulated as a 3d cube around the (0,0,0) At the click of a button I want to sample N random points in the image and get the distance of these points from the camera (distance in the "real world"). So far I've managed to create the scene of the moving camera and the cube (Example)
我尝试了 gluUnProject 来获得 3d 点
I tried gluUnProject to get the 3d point
model_view = np.array(glGetDoublev(GL_MODELVIEW_MATRIX))
proj = np.array(glGetDoublev(GL_PROJECTION_MATRIX))
view = np.array(glGetDoublev(GL_VIEWPORT))
3d_point = gluUnProject(x,y, 0.0)
其中 x,y 是图像中像素的坐标，但是当我检查知道它们位置(立方角)的像素时，我得到的感觉像是随机结果.
where x,y are the coordinates of the pixel in the image, but when I check this on pixels that I know their location (the cube corners) I get what feels like random results.
我对 openGL 非常陌生，所以我可能会遗漏一些东西，从数学角度来说，我想做的就是在像素坐标上应用投影和视图矩阵的倒数，但这不起作用.
I very new to openGL so I might be missing something, math-wise all I want to do is apply the inverse of the projection and view matrix on the pixel coordinates but that doesnt work.
我在下面附上了房间模拟的代码.
I attache the code for the room simulation below.
提前致谢.
import pygame
from pygame.locals import *
import numpy as np
import random
from OpenGL.GL import *
from OpenGL.GLU import *
display = (800, 600)
import math
def get_cube_information():
vertices = (
(1, -1, -1),
(1, 1, -1),
(-1, 1, -1),
(-1, -1, -1),
(1, -1, 1),
(1, 1, 1, ),
(-1, -1, 1),
(-1, 1, 1),
)
edges = (
(0,1),
(0,3),
(0,4),
(2,1),
(2,3),
(2,7),
(6,3),
(6,4),
(6,7),
(5,1),
(5,4),
(5,7),
)
surfaces = (
(0,1,2,3),
(3,2,7,6),
(6,7,5,4),
(4,5,1,0),
(1,5,7,2),
(4,0,3,6),
)
colors = (
(1.000, 0.920, 0.000),
(0.000, 0.860, 0.000),
(1.000, 0.480, 0.000),
(1.000, 1.000, 1.000),
(0.900, 0.000, 0.000),
(0.000, 0.000, 0.950)
)
return vertices, edges, surfaces, colors
def Cube():
glBegin(GL_QUADS)
(vertices, edges, surfaces, colors) = get_cube_information()
for i, surface in enumerate(surfaces):
x = 0
color = colors[i]
for vertex in surface:
x += 1
glColor3fv(color)
glVertex3fv(vertices[vertex])
glEnd()
glBegin(GL_LINES)
for edge in edges:
for vertex in edge:
glVertex3fv(vertices[vertex])
glEnd()
def main():
pygame.init()
tx = 0
ty = 0
tz = 0
ry = 0
rx = 0
pygame.display.set_mode(display, DOUBLEBUF|OPENGL|RESIZABLE)
glMatrixMode(GL_PROJECTION)
gluPerspective(45, (display[0] / display[1]), 0.1, 50.0)
view_mat = np.matrix(np.identity(4), copy=False, dtype='float32')
glMatrixMode(GL_MODELVIEW)
glLoadIdentity()
glTranslatef(0, 0, 0)
glGetFloatv(GL_MODELVIEW_MATRIX, view_mat)
glLoadIdentity()
while True:
for event in pygame.event.get():
if event.type == pygame.QUIT:
pygame.quit()
quit()
elif event.type == pygame.KEYDOWN:
if event.key == pygame.K_ESCAPE:
pygame.quit()
quit()
if event.key == pygame.K_a:
tx = 0.05
elif event.key == pygame.K_d:
tx = -0.05
elif event.key == pygame.K_w:
tz = 0.05
elif event.key == pygame.K_s:
tz = -0.05
elif event.key == pygame.K_RIGHT:
ry = 1.0
elif event.key == pygame.K_LEFT:
ry = -1.0
elif event.key == pygame.K_UP:
rx = -1.0
elif event.key == pygame.K_DOWN:
rx = 1.0
elif event.key == pygame.K_SPACE:
continue
elif event.type == pygame.KEYUP:
if event.key == pygame.K_a and tx > 0:
tx = 0
elif event.key == pygame.K_d and tx < 0:
tx = 0
elif event.key == pygame.K_w and tz > 0:
tz = 0
elif event.key == pygame.K_s and tz < 0:
tz = 0
elif event.key == pygame.K_RIGHT and ry > 0:
ry = 0.0
elif event.key == pygame.K_LEFT and ry < 0:
ry = 0.0
elif event.key == pygame.K_DOWN and rx > 0:
rx = 0.0
elif event.key == pygame.K_UP and rx < 0:
rx = 0.0
elif event.type == pygame.MOUSEBUTTONDOWN:
#here I want to sample the points and return their (x,y) in the image and their distance from the camera.
continue
glPushMatrix()
glLoadIdentity()
glTranslatef(tx, ty, tz)
glRotatef(ry, 0, 1, 0)
glRotatef(rx, 1, 0, 0)
glMultMatrixf(view_mat)
glGetFloatv(GL_MODELVIEW_MATRIX, view_mat)
glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT)
Cube()
glPopMatrix()
pygame.display.flip()
pygame.time.wait(10)
main()
推荐答案
要在视口中找到一个点的世界位置，您必须知道该点的深度值.
To find the world position of a point on the viewport, you have to know the depth value of the point.
x 和 y 屏幕位置和深度已转换为 [-1, 1] 范围内的标准化设备坐标.为此，必须知道视口矩形:
The x and y screen position and the depth have be transformed to normalized device coordinates in the range [-1, 1]. For this the viewport rectangle has to be known:
ndc = [2.0* x/vp_width - 1.0, 1.0 - 2.0*y/vp_height, depth*2.0 - 1.0];
归一化的设备空间坐标必须通过逆投影矩阵转换到视图空间(最后必须执行透视划分).
The normalized device space coordinate has to be transformed by the inverse projection matrix to the view space (Finally a perspective divide has to be performed).
使用逆视图矩阵，可以将视图空间坐标转换为世界空间.
With the inverse view matrix, the view space coordinate can be transformed to world space.
gluUnProject 为您完成所有这些，但您必须知道片段的深度.片段的深度可以通过 读取glReadPixels:
# get mouse position
x, y = pygame.mouse.get_pos()
# get the fragment depth
depth = glReadPixels(x, y, 1, 1, GL_DEPTH_COMPONENT, GL_FLOAT)
# get projection matrix, view matrix and the viewport rectangle
model_view = np.array(glGetDoublev(GL_MODELVIEW_MATRIX))
proj = np.array(glGetDoublev(GL_PROJECTION_MATRIX))
view = np.array(glGetIntegerv(GL_VIEWPORT))
# unproject the point
point = gluUnProject(x, y, depth, model_view, proj, view)
print( point )
<小时>
注意，您必须启用深度测试，否则深度缓冲区将不会被设置.这也带来了好处，前面的多边形覆盖了它们后面"的多边形:
Note, you have to enable the Depth Test otherwise the depth buffer will not be set. This also gives the benefit, that polygons at the front cover the polygons "behind" them:
glEnable(GL_DEPTH_TEST)
Cube()
<小时>
当然投影矩阵和模型视图矩阵必须正确设置，当vlues被glGetDoublev(GL_PROJECTION_MATRIX)分别读取glGetDoublev(GL_MODELVIEW_MATRIX)时.
这意味着视图矩阵的读取要在设置之后进行:
This means that the reading of the view matrix should be done after setting it:
glPushMatrix()
glLoadIdentity()
glTranslatef(tx, ty, tz)
glRotatef(ry, 0, 1, 0)
glRotatef(rx, 1, 0, 0)
glMultMatrixf(view_mat)
glGetFloatv(GL_MODELVIEW_MATRIX, view_mat)
model_view = np.array(glGetDoublev(GL_MODELVIEW_MATRIX))
注意，如果gluUnProject的第4个参数(model)使用单位矩阵，则gluUnProject不计算世界坐标，但它计算视图坐标.
Note, if for the 4th parameter (model) of gluUnProject is used the identity matrix, then gluUnProject does not calculate world coordinates, but it calculates view coordinates.
这篇关于在python opengl中使用图像中的2d点获取空间中的3d点的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
