具有高多边形网格的 OpenGL 3D 光线拾取 [英] OpenGL 3D-raypicking with high poly meshes
问题描述
如何在包含高多边形网格的模型的 3d 场景中实现 3d 光线拾取?
How to implement 3d raypicking in an 3d scene with models that contain high poly meshes?
迭代所有三角形以执行三角形线相交测试需要太多时间.我知道存在八叉树等方法，应该可以将它们用于场景中的模型，但我不知道我应该如何在网格级别使用这些概念.但是，如果您在网格级别使用八叉树，应该如何解决超出八叉树体积边界的多边形问题?
It takes too much time to iterate over all triangles to perform a triangle-line-intersection test. I know that there exist methods like octree etc. and it should be possible to use these for the models in the scene, but I do not know how I should use these concepts at mesh-level. But if you use an octree at mesh-level, how should one cover problems with polygons, that exceed the boundaries of the octree volumes?
对于用于实时 OpenGL 应用程序的高多边形模型的 3d 射线交叉，您有什么建议吗?
Do you have any advice which method is suitable or recommended for 3d ray-intersections with high poly models for real-time OpenGl applications?
推荐答案
对于光线拾取渲染对象(例如通过鼠标)，最好的选择是使用已经渲染的缓冲区，因为与光线相比，读取它们的成本非常低复杂场景的交叉测试.我们的想法是将每个可拾取的渲染对象渲染为每个您需要的关于它们的信息的单独缓冲区，例如:
For ray picking rendered objects (like by mouse) the best option is to use the already rendered buffers as there is very little cost of reading them in comparison to ray intersection tests on complex scene. The idea is to render each pick-able rendered object to separate buffer per each info you need about them for example like this:
深度缓冲
这将为您提供光线与对象相交的 3D 位置.
this will give you the 3D position of the ray intersection with object.
模板缓冲区
如果每个对象都使用其 ID(或其在对象列表中的索引)渲染到模板，那么您可以直接获取所选对象.
if each object rendered to stencil with its ID (or its index in object list) then you can get the picked object directly.
任何其他
还有辅助颜色附件和 FBO.因此，您可以添加任何其他内容，例如法线向量或您需要的任何内容.
there are also secondary color attachments and FBO's out there. So you can add any other stuff like normal vector or what ever you need.
如果编码正确，所有这些只会略微降低性能(甚至根本不会)，因为您不需要计算任何东西，它只是每个缓冲区每个片段的一次写入.
If coded right all of this will reduce performance only slightly (even not at all) as you do not need to compute anything its just a single write per fragment per buffer.
选择本身很容易，您只需从所需的所有缓冲区中读取相应的像素并将其转换为所需的格式即可.
The picking itself is easy you just read the corresponding pixel from all the buffers you need and convert to wanted format.
这里使用固定管道(无着色器)的简单 C++/VCL 示例...
Here simple C++/VCL example using fixed pipeline (no shaders)...
//---------------------------------------------------------------------------
#include <vcl.h>
#include <math.h>
#pragma hdrstop
#include "Unit1.h"
#include "gl_simple.h"
//---------------------------------------------------------------------------
#pragma package(smart_init)
#pragma resource "*.dfm"
TForm1 *Form1;
//---------------------------------------------------------------------------
void matrix_mul_vector(double *c,double *a,double *b,double w=1.0)
{
double q[3];
q[0]=(a[ 0]*b[0])+(a[ 4]*b[1])+(a[ 8]*b[2])+(a[12]*w);
q[1]=(a[ 1]*b[0])+(a[ 5]*b[1])+(a[ 9]*b[2])+(a[13]*w);
q[2]=(a[ 2]*b[0])+(a[ 6]*b[1])+(a[10]*b[2])+(a[14]*w);
for(int i=0;i<3;i++) c[i]=q[i];
}
//---------------------------------------------------------------------------
class glMouse
{
public:
int sx,sy; // framebuffer position [pixels]
double pos[3]; // [GCS] ray end coordinate (or z_far)
double beg[3]; // [GCS] ray start (z_near)
double dir[3]; // [GCS] ray direction
double depth; // [GCS] perpendicular distance to camera
WORD id; // selected object id
double x0,y0,xs,ys,zFar,zNear; // viewport and projection
double *eye; // camera direct matrix pointer
double fx,fy; // perspective scales
glMouse(){ eye=NULL; for (int i=0;i<3;i++) { pos[i]=0.0; beg[i]=0.0; dir[i]=0.0; } id=0; x0=0.0; y0=0.0; xs=0.0; ys=0.0; fx=0.0; fy=0.0; depth=0.0; }
glMouse(glMouse& a){ *this=a; };
~glMouse(){};
glMouse* operator = (const glMouse *a) { *this=*a; return this; };
// glMouse* operator = (const glMouse &a) { ...copy... return this; };
void resize(double _x0,double _y0,double _xs,double _ys,double *_eye)
{
double per[16];
x0=_x0; y0=_y0; xs=_xs; ys=_ys; eye=_eye;
glGetDoublev(GL_PROJECTION_MATRIX,per);
zFar =0.5*per[14]*(1.0-((per[10]-1.0)/(per[10]+1.0)));
zNear=zFar*(per[10]+1.0)/(per[10]-1.0);
fx=per[0];
fy=per[5];
}
void pick(double x,double y) // test screen x,y [pixels] position
{
int i;
double l;
GLfloat _z;
GLint _id;
sx=x; sy=ys-1.0-y;
// read depth z and linearize
glReadPixels(sx,sy,1,1,GL_DEPTH_COMPONENT,GL_FLOAT,&_z);// read depth value
depth=_z; // logarithmic
depth=(2.0*depth)-1.0; // logarithmic NDC
z=(2.0*zNear*zFar)/(zFar+zNear-(z*(zFar-zNear))); // linear <zNear,zFar>
// read object ID
glReadPixels(sx,sy,1,1,GL_STENCIL_INDEX,GL_INT,&_id); // read stencil value
id=_id;
// win [pixel] -> GL NDC <-1,+1>
x= (2.0*(x-x0)/xs)-1.0;
y=1.0-(2.0*(y-y0)/ys);
// ray start GL camera [LCS]
beg[2]=-zNear;
beg[1]=(-beg[2]/fy)*y;
beg[0]=(-beg[2]/fx)*x;
// ray direction GL camera [LCS]
for (l=0.0,i=0;i<3;i++) l+=beg[i]*beg[i]; l=1.0/sqrt(l);
for (i=0;i<3;i++) dir[0]=beg[0]*l;
// ray end GL camera [LCS]
pos[2]=-depth;
pos[1]=(-pos[2]/fy)*y;
pos[0]=(-pos[2]/fx)*x;
// convert to [GCS]
matrix_mul_vector(beg,eye,beg);
matrix_mul_vector(pos,eye,pos);
matrix_mul_vector(dir,eye,dir,0.0);
}
};
//---------------------------------------------------------------------------
// camera & mouse
double eye[16],ieye[16]; // direct view,inverse view and perspective matrices
glMouse mouse;
// objects
struct object
{
WORD id; // unique non zero ID
double m[16]; // direct model matrix
object(){}; object(object& a){ *this=a; }; ~object(){}; object* operator = (const object *a) { *this=*a; return this; }; /*object* operator = (const object &a) { ...copy... return this; };*/
};
const int objs=7;
object obj[objs];
// textures
GLuint txr=-1;
//---------------------------------------------------------------------------
void matrix_inv(double *a,double *b) // a[16] = Inverse(b[16])
{
double x,y,z;
// transpose of rotation matrix
a[ 0]=b[ 0];
a[ 5]=b[ 5];
a[10]=b[10];
x=b[1]; a[1]=b[4]; a[4]=x;
x=b[2]; a[2]=b[8]; a[8]=x;
x=b[6]; a[6]=b[9]; a[9]=x;
// copy projection part
a[ 3]=b[ 3];
a[ 7]=b[ 7];
a[11]=b[11];
a[15]=b[15];
// convert origin: new_pos = - new_rotation_matrix * old_pos
x=(a[ 0]*b[12])+(a[ 4]*b[13])+(a[ 8]*b[14]);
y=(a[ 1]*b[12])+(a[ 5]*b[13])+(a[ 9]*b[14]);
z=(a[ 2]*b[12])+(a[ 6]*b[13])+(a[10]*b[14]);
a[12]=-x;
a[13]=-y;
a[14]=-z;
}
//---------------------------------------------------------------------------
void gl_draw()
{
int i; object *o;
double a;
glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT | GL_STENCIL_BUFFER_BIT );
glEnable(GL_CULL_FACE);
glEnable(GL_DEPTH_TEST);
glEnable(GL_STENCIL_TEST);
glStencilOp(GL_KEEP, GL_KEEP, GL_REPLACE);
glStencilMask(0xFFFF); // Write to stencil buffer
glStencilFunc(GL_ALWAYS,0,0xFFFF); // Set any stencil to 0
for (o=obj,i=0;i<objs;i++,o++)
{
glMatrixMode(GL_MODELVIEW);
glLoadMatrixd(ieye);
glMultMatrixd(o->m);
glStencilFunc(GL_ALWAYS,o->id,0xFFFF); // Set any stencil to object ID
vao_draw();
}
glStencilFunc(GL_ALWAYS,0,0xFFFF); // Set any stencil to 0
glDisable(GL_STENCIL_TEST); // no need fot testing
// render mouse
glMatrixMode(GL_MODELVIEW);
glLoadMatrixd(ieye);
a=0.1*mouse.depth;
glColor3f(0.0,1.0,0.0);
glBegin(GL_LINES);
glVertex3d(mouse.pos[0]+a,mouse.pos[1],mouse.pos[2]);
glVertex3d(mouse.pos[0]-a,mouse.pos[1],mouse.pos[2]);
glVertex3d(mouse.pos[0],mouse.pos[1]+a,mouse.pos[2]);
glVertex3d(mouse.pos[0],mouse.pos[1]-a,mouse.pos[2]);
glVertex3d(mouse.pos[0],mouse.pos[1],mouse.pos[2]+a);
glVertex3d(mouse.pos[0],mouse.pos[1],mouse.pos[2]-a);
glEnd();
Form1->Caption=AnsiString().sprintf("%.3lf , %.3lf , %.3lf : %u",mouse.pos[0],mouse.pos[1],mouse.pos[2],mouse.id);
// debug buffer views
if ((Form1->ck_depth->Checked)||(Form1->ck_stencil->Checked))
{
glDisable(GL_DEPTH_TEST);
glMatrixMode(GL_PROJECTION);
glPushMatrix();
glLoadIdentity();
glMatrixMode(GL_MODELVIEW);
glLoadIdentity();
glEnable(GL_TEXTURE_2D);
glBindTexture(GL_TEXTURE_2D,txr);
GLfloat *f=new GLfloat[xs*ys],z;
if (Form1->ck_depth ->Checked)
{
glReadPixels(0,0,xs,ys,GL_DEPTH_COMPONENT,GL_FLOAT,f);
for (i=0;i<xs*ys;i++) f[i]=1.0-(2.0*mouse.zNear)/(mouse.zFar+mouse.zNear-(((2.0*f[i])-1.0)*(mouse.zFar-mouse.zNear)));
glTexImage2D(GL_TEXTURE_2D, 0, GL_RGB, xs, ys, 0, GL_RED, GL_FLOAT, f);
}
if (Form1->ck_stencil->Checked)
{
glReadPixels(0,0,xs,ys,GL_STENCIL_INDEX,GL_FLOAT,f);
for (i=0;i<xs*ys;i++) f[i]/=float(objs);
glTexImage2D(GL_TEXTURE_2D, 0, GL_RGB, xs, ys, 0, GL_GREEN, GL_FLOAT, f);
}
delete[] f;
glColor3f(1.0,1.0,1.0);
glBegin(GL_QUADS);
glTexCoord2f(1.0,0.0); glVertex2f(+1.0,-1.0);
glTexCoord2f(1.0,1.0); glVertex2f(+1.0,+1.0);
glTexCoord2f(0.0,1.0); glVertex2f(-1.0,+1.0);
glTexCoord2f(0.0,0.0); glVertex2f(-1.0,-1.0);
glEnd();
glMatrixMode(GL_PROJECTION);
glPopMatrix();
glDisable(GL_TEXTURE_2D);
glEnable(GL_DEPTH_TEST);
}
glFlush();
SwapBuffers(hdc);
}
//---------------------------------------------------------------------------
__fastcall TForm1::TForm1(TComponent* Owner):TForm(Owner)
{
int i;
object *o;
gl_init(Handle);
vao_init();
// init textures
glGenTextures(1,&txr);
glEnable(GL_TEXTURE_2D);
glBindTexture(GL_TEXTURE_2D,txr);
glPixelStorei(GL_UNPACK_ALIGNMENT, 4);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S,GL_CLAMP_TO_EDGE);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T,GL_CLAMP_TO_EDGE);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER,GL_NEAREST);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER,GL_NEAREST);
glTexEnvf(GL_TEXTURE_ENV, GL_TEXTURE_ENV_MODE,GL_COPY);
glDisable(GL_TEXTURE_2D);
// init objects
glMatrixMode(GL_MODELVIEW);
glLoadIdentity();
glTranslatef(-1.5,4.7,-8.0);
for (o=obj,i=0;i<objs;i++,o++)
{
o->id=i+1; // unique non zero ID
glGetDoublev(GL_MODELVIEW_MATRIX,o->m);
glRotatef(360.0/float(objs),0.0,0.0,1.0);
glTranslatef(-3.0,0.0,0.0);
}
for (o=obj,i=0;i<objs;i++,o++)
{
glLoadMatrixd(o->m);
glRotatef(180.0*Random(),Random(),Random(),Random());
glGetDoublev(GL_MODELVIEW_MATRIX,o->m);
}
}
//---------------------------------------------------------------------------
void __fastcall TForm1::FormDestroy(TObject *Sender)
{
glDeleteTextures(1,&txr);
gl_exit();
vao_exit();
}
//---------------------------------------------------------------------------
void __fastcall TForm1::FormResize(TObject *Sender)
{
gl_resize(ClientWidth,ClientHeight);
// obtain/init matrices
glMatrixMode(GL_MODELVIEW);
glLoadIdentity();
glTranslatef(0,0,-15.0);
glGetDoublev(GL_MODELVIEW_MATRIX,ieye);
matrix_inv(eye,ieye);
mouse.resize(0,0,xs,ys,eye);
}
//---------------------------------------------------------------------------
void __fastcall TForm1::FormPaint(TObject *Sender)
{
gl_draw();
}
//---------------------------------------------------------------------------
void __fastcall TForm1::Timer1Timer(TObject *Sender)
{
gl_draw();
}
//---------------------------------------------------------------------------
void __fastcall TForm1::FormMouseWheel(TObject *Sender, TShiftState Shift, int WheelDelta, TPoint &MousePos, bool &Handled)
{
GLfloat dz=2.0;
if (WheelDelta<0) dz=-dz;
glMatrixMode(GL_MODELVIEW);
glLoadMatrixd(ieye);
glTranslatef(0,0,dz);
glGetDoublev(GL_MODELVIEW_MATRIX,ieye);
matrix_inv(eye,ieye);
gl_draw();
}
//---------------------------------------------------------------------------
void __fastcall TForm1::FormMouseMove(TObject *Sender, TShiftState Shift, int X, int Y)
{
mouse.pick(X,Y);
}
//---------------------------------------------------------------------------
void __fastcall TForm1::ck_depthClick(TObject *Sender)
{
gl_draw();
}
//---------------------------------------------------------------------------
这里是从左 RGB、Depth、Stencil 预览:
这里是捕获的 GIF:
前3个数字是[GCS]中选取像素的3D位置，标题中的最后一个数字是选取的ID，其中0 表示没有对象.
the first 3 numbers are the 3D position of picked pixel in [GCS] and the last number in caption is the picked ID where 0 means no object.
该示例使用来自此处的 gl_simple,h:
The example is using gl_simple,h from here:
您可以忽略 VCL 的内容，因为将事件移植到您的环境中并不重要...
You can ignore the VCL stuff as its not important just port the events to your environment...
那该怎么办:
渲染
您需要将模板缓冲区添加到您的 GL 窗口像素格式，因此在我的情况下，我只需添加:
You need add stencil buffer to your GL window pixel format so in my case I just add:
pfd.cStencilBits = 16;
从gl_simple.h 进入gl_init() 函数.还将其位添加到 glClear 中，并将每个对象模板设置为其 ID，就像我在 gl_draw() 中所做的那样.
into gl_init() function from gl_simple.h. Also add its bit into glClear and set each objects stencil to its ID Like I did in gl_draw().
采摘
我编写了一个小的 glMouse 类来完成所有繁重的工作.每次改变视角、视图或视口时，调用它的 glMouse::resize 函数.这将准备稍后计算所需的所有常量.当心它需要直接的相机/视图矩阵！！！
I wrote a small glMouse class that do all the heavy lifting. On each change of perspective, view, or viewport call its glMouse::resize function. That will prepare all the constants needed for the computations later. Beware it needs direct camera/view matrix !!!
现在在每次鼠标移动(或单击或其他)时调用 glMouse::pick 函数，然后使用像 id 这样的结果，它将返回 ID 选取的对象是用 或 pos 渲染的，它是光线对象的全局世界坐标 ([GCS]) 中的 3D 坐标十字路口.
Now on each mouse movement (or click or whatever) call the glMouse::pick function and then use the results like id which will return the ID picked object was rendered with or pos which is the 3D coordinate in global world coordinates ([GCS]) of the ray object intersection.
该函数仅读取深度和模板缓冲区.像这样线性化深度:
The function just read the depth and stencil buffers. Linearize depth like here:
并计算[GCS]中的光线beg,dir,pos,depth.
正常
您有 2 个选项，要么将法线渲染为另一个最简单、最精确的缓冲区.或者读取所选像素周围 2 个或更多相邻像素的深度，计算它们的 3D 位置.如果需要，使用叉积计算您的正常值和平均值.但这可能会导致边缘出现伪影.
You got 2 options either render your normal as another buffer which is the simplest and most precise. Or read depths of 2 or more neighboring pixels around picked one compute their 3D positions. From that using cross product compute you normal(s) and average if needed. But this can lead to artifacts on edges.
正如评论中提到的，为了提高准确性，您应该使用线性深度缓冲区而不是像这样的线性化对数:
As mentioned in the comments to boost accuracy you should use linear depth buffer instead of linearized logarithmic like this:
顺便说一句，我在这里使用了相同的技术(在基于 GDI 的 SW 等距渲染中):
Btw I used the same technique in here (in GDI based SW isometric render):
[Edit1] 8 位模板缓冲区
现在可靠的模板位宽只有 8 位，这将 id 的数量限制为 255.这在大多数情况下是不够的.一种解决方法是将索引渲染为颜色，然后将帧存储到 CPU 内存中，然后正常渲染颜色.然后在需要时使用存储的框架进行拣选.也可以渲染到纹理或颜色附件.
Well these days the reliable stencil bitwidth is only 8bit which limits the number of ids to 255. That is in most cases not enough. A workaround is to render the indexes as colors then store the frame into CPU memory and then render colors normaly. Then when needed using the stored frame for picking. Rendering to texture or color attachment is also a possibility.
[Edit2] 一些相关链接
这篇关于具有高多边形网格的 OpenGL 3D 光线拾取的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
