GPU拾取-子画面周围的不可见像素 [英] gpu picking - invisible pixels around sprites
问题描述
我正在渲染一个包含精灵的采摘场景.当我的光标靠近精灵时，它注册为一种颜色并被选中".放大小精灵时，此不可见的边框会变大.
I'm rendering a picking scene that contains sprites. As my cursor gets close to the sprite, it registers as a color and gets "picked". This invisible border gets larger closer you zoom into the sprites.
打开控制台以查看实时打印的ID.将光标移近或移远，移到大和小的精灵上.您会看到精灵在不可见的边框上被选中.普通的几何图形不会发生这种行为，只是精灵会发生这种情况.
Open up your console to see the IDs printed in real time. Move your cursor closer and further away to large and small sprites. You'll see that sprites get selected on an invisible border. This behavior does not happen with regular geometry, just with sprites.
这很奇怪，因为我渲染了renderer.readRenderTargetPixels实际看到的内容.
It's weird because I'm rendering out what renderer.readRenderTargetPixels actually sees.
如何摆脱看不见的边框，以进行更准确的拾取?
How can I get rid of the invisible borders for more accurate picking?
var renderer, scene, camera, controls;
var particles, uniforms;
var PARTICLE_SIZE = 50;
var raycaster, intersects;
var mouse, INTERSECTED;
var pickingTexture;
var numOfVertices;
init();
animate();
function init() {
container = document.getElementById('container');
scene = new THREE.Scene();
camera = new THREE.PerspectiveCamera(45, window.innerWidth / window.innerHeight, 1, 10000);
camera.position.z = 150;
//
var geometry1 = new THREE.BoxGeometry(200, 200, 200, 4, 4, 4);
var vertices = geometry1.vertices;
numOfVertices = vertices.length;
var positions = new Float32Array(vertices.length * 3);
var colors = new Float32Array(vertices.length * 3);
var sizes = new Float32Array(vertices.length);
var vertex;
var color = new THREE.Color();
for (var i = 0, l = vertices.length; i < l; i++) {
vertex = vertices[i];
vertex.toArray(positions, i * 3);
color.setHex(i + 1);
color.toArray(colors, i * 3);
sizes[i] = PARTICLE_SIZE * 0.5;
}
var geometry = new THREE.BufferGeometry();
geometry.addAttribute('position', new THREE.BufferAttribute(positions, 3));
geometry.addAttribute('customColor', new THREE.BufferAttribute(colors, 3));
geometry.addAttribute('size', new THREE.BufferAttribute(sizes, 1));
//
var material = new THREE.ShaderMaterial({
uniforms: {
// texture: {type: "t", value: THREE.ImageUtils.loadTexture("../textures/circle.png")}
texture: {type: "t", value: THREE.ImageUtils.loadTexture("../textures/disc.png")}
},
vertexShader: document.getElementById('vertexshader').textContent,
fragmentShader: document.getElementById('fragmentshader').textContent,
depthTest: false,
transparent: false
// alphaTest: 0.9
});
//
particles = new THREE.Points(geometry, material);
scene.add(particles);
//
renderer = new THREE.WebGLRenderer({
antialias: true,
alpha: true
});
renderer.setPixelRatio(window.devicePixelRatio);
renderer.setSize(window.innerWidth, window.innerHeight);
renderer.setClearColor(0xffffff);
container.appendChild(renderer.domElement);
//
raycaster = new THREE.Raycaster();
mouse = new THREE.Vector2();
//
//
window.addEventListener('resize', onWindowResize, false);
document.addEventListener('mousemove', onDocumentMouseMove, false);
// defaults are on the right (except minFilter)
var options = {
format: THREE.RGBAFormat, // THREE.RGBAFormat
type: THREE.UnsignedByteType, // THREE.UnsignedByteType
anisotropy: 1, // 1
magFilter: THREE.LinearFilter, // THREE.LinearFilter
minFilter: THREE.LinearFilter, // THREE.LinearFilter
depthBuffer: true, // true
stencilBuffer: true // true
};
pickingTexture = new THREE.WebGLRenderTarget(window.innerWidth, window.innerHeight, options);
pickingTexture.texture.generateMipmaps = false;
controls = new THREE.OrbitControls(camera, container);
controls.damping = 0.2;
controls.enableDamping = false;
}
function onDocumentMouseMove(e) {
// event.preventDefault();
mouse.x = e.clientX;
mouse.y = e.clientY;
}
function onWindowResize() {
camera.aspect = window.innerWidth / window.innerHeight;
camera.updateProjectionMatrix();
renderer.setSize(window.innerWidth, window.innerHeight);
}
function animate() {
requestAnimationFrame(animate);
controls.update();
render();
}
function render() {
pick();
renderer.render(scene, camera);
}
function pick() {
renderer.render(scene, camera, pickingTexture);
//create buffer for reading single pixel
var pixelBuffer = new Uint8Array(4);
//read the pixel under the mouse from the texture
renderer.readRenderTargetPixels(pickingTexture, mouse.x, pickingTexture.height - mouse.y, 1, 1, pixelBuffer);
//interpret the pixel as an ID
var id = ( pixelBuffer[0] << 16 ) | ( pixelBuffer[1] << 8 ) | ( pixelBuffer[2] );
if (id <= numOfVertices) console.log(id);
}
body {
color: #ffffff;
background-color: #000000;
margin: 0px;
overflow: hidden;
}
<script src="http://threejs.org/build/three.min.js"></script>
<script src="http://threejs.org/examples/js/controls/OrbitControls.js"></script>
<script type="x-shader/x-fragment" id="fragmentshader">
uniform sampler2D texture;
varying vec3 vColor;
void main() {
// solid squares of color
gl_FragColor = vec4( vColor, 1.0 );
}
</script>
<script type="x-shader/x-vertex" id="vertexshader">
attribute float size;
attribute vec3 customColor;
varying vec3 vColor;
void main() {
vColor = customColor;
vec4 mvPosition = modelViewMatrix * vec4( position, 1.0 );
gl_PointSize = size * ( 300.0 / length( mvPosition.xyz ) );
gl_Position = projectionMatrix * mvPosition;
}
</script>
<div id="container"></div>
推荐答案
问题是您所使用的设备具有devicePixelRatio！= 1.0和three.js大小.
The problem is you're on a device that has a devicePixelRatio != 1.0 and three.js lying about the size.
因为您叫renderer.setPixelRatio，所以魔术在幕后发生.您画布的大小不是您要求的大小，而是根据three.js代码中隐藏的某些公式得出的其他大小.
Because you called renderer.setPixelRatio now magic happens behind the scenes. Your canvas is not the size you requested it's some other size based on some formula hidden in the three.js code.
那么，会发生什么.您的画布是一种尺寸，但是渲染目标是另一种尺寸.着色器使用gl_PointSize绘制其点.该尺寸以设备像素为单位.由于渲染目标的大小不同，因此渲染目标中点的大小与屏幕上的点不同.
So, what happens. Your canvas is one size but your render target is a different size. Your shader uses gl_PointSize to draw its points. That size is in device pixels. Because your render target is a different size the size of the points are different in your render target than they are on screen.
将呼叫移至render.setPixelRatio，它将开始工作.
Remove the call to render.setPixelRatio and it will start working.
IMO纠正此问题的正确方法是自己使用devicePixelRatio，因为这样您可以100％看到所有发生的事情.幕后没有发生魔法.
IMO the correct way to fix this is to use devicePixelRatio yourself because that way everything that is happening is 100% visible to you. No magic happening behind the scenes.
所以
-
摆脱容器并直接使用画布
Get rid of the container and use a canvas directly
<canvas id="c"></canvas>
将画布设置为使用100vw表示宽度，使用100vh表示高度，并将主体设置为margin: 0;
set the canvas to use 100vw for width, 100vh for height and made the body margin: 0;
canvas { width: 100vw; height: 100vh; display: block; }
body { margin: 0; }
这将使您的画布自动拉伸以填充窗口.
This will make your canvas stretch automatically to fill the window.
使用浏览器拉伸画布的大小选择其drawingBuffer应为的大小，然后乘以devicePixelRatio.假设您实际上要支持设备像素比率.无需重复执行两次，因此请遵循 D.R.Y. ，只需在onWindowResize中进行操作即可.
Use the size the browser stretched the canvas to choose the size its drawingBuffer should be and multiply by devicePixelRatio. That assumes you actually want to support device pixel ratio. No need to do this twice so following D.R.Y. so just do it in onWindowResize.
canvas = document.getElementById("c");
renderer = new THREE.WebGLRenderer({
antialias: true,
alpha: true,
canvas: canvas,
});
pickingTexture = new THREE.WebGLRenderTarget(1, 1, options);
onWindowResize();
...
function onWindowResize() {
var width = canvas.clientWidth * window.devicePixelRatio;
var height = canvas.clientHeight * window.devicePixelRatio;
camera.aspect = width / height;
camera.updateProjectionMatrix();
renderer.setSize(width, height, false); // YOU MUST PASS FALSE HERE otherwise three.js will muck with the CSS
pickingTexture.setSize(width, height);
}
将鼠标坐标转换为设备坐标
Convert the mouse coordinates into device coordinates
renderer.readRenderTargetPixels(
pickingTexture,
mouse.x * window.devicePixelRatio,
pickingTexture.height - mouse.y * window.devicePixelRatio,
1, 1, pixelBuffer);
这就是解决方案
var renderer, scene, camera, controls;
var particles, uniforms;
var PARTICLE_SIZE = 50;
var raycaster, intersects;
var mouse, INTERSECTED;
var pickingTexture;
var numOfVertices;
var info = document.querySelector('#info');
init();
animate();
function init() {
canvas = document.getElementById('c');
scene = new THREE.Scene();
camera = new THREE.PerspectiveCamera(45, 1, 1, 10000);
camera.position.z = 150;
//
var geometry1 = new THREE.BoxGeometry(200, 200, 200, 4, 4, 4);
var vertices = geometry1.vertices;
numOfVertices = vertices.length;
var positions = new Float32Array(vertices.length * 3);
var colors = new Float32Array(vertices.length * 3);
var sizes = new Float32Array(vertices.length);
var vertex;
var color = new THREE.Color();
for (var i = 0, l = vertices.length; i < l; i++) {
vertex = vertices[i];
vertex.toArray(positions, i * 3);
color.setHex(i + 1);
color.toArray(colors, i * 3);
sizes[i] = PARTICLE_SIZE * 0.5;
}
var geometry = new THREE.BufferGeometry();
geometry.setAttribute('position', new THREE.BufferAttribute(positions, 3));
geometry.setAttribute('customColor', new THREE.BufferAttribute(colors, 3));
geometry.setAttribute('size', new THREE.BufferAttribute(sizes, 1));
//
var loader = new THREE.TextureLoader();
var material = new THREE.ShaderMaterial({
uniforms: {
// texture: {type: "t", value: THREE.ImageUtils.loadTexture("../textures/circle.png")}
texture: {value: loader.load("https://i.imgur.com/iXT97XR.png")}
},
vertexShader: document.getElementById('vertexshader').textContent,
fragmentShader: document.getElementById('fragmentshader').textContent,
depthTest: false,
transparent: false
// alphaTest: 0.9
});
//
particles = new THREE.Points(geometry, material);
scene.add(particles);
//
renderer = new THREE.WebGLRenderer({
antialias: true,
alpha: true,
canvas: canvas,
});
renderer.setClearColor(0xffffff);
//
raycaster = new THREE.Raycaster();
mouse = new THREE.Vector2();
//
//
window.addEventListener('resize', onWindowResize, false);
document.addEventListener('mousemove', onDocumentMouseMove, false);
// defaults are on the right (except minFilter)
var options = {
format: THREE.RGBAFormat, // THREE.RGBAFormat
type: THREE.UnsignedByteType, // THREE.UnsignedByteType
anisotropy: 1, // 1
magFilter: THREE.LinearFilter, // THREE.LinearFilter
minFilter: THREE.LinearFilter, // THREE.LinearFilter
depthBuffer: true, // true
stencilBuffer: true // true
};
pickingTexture = new THREE.WebGLRenderTarget(1, 1, options);
pickingTexture.texture.generateMipmaps = false;
controls = new THREE.OrbitControls(camera, canvas);
controls.damping = 0.2;
controls.enableDamping = false;
onWindowResize();
}
function onDocumentMouseMove(e) {
// event.preventDefault();
mouse.x = e.clientX;
mouse.y = e.clientY;
}
function onWindowResize() {
var width = canvas.clientWidth * window.devicePixelRatio;
var height = canvas.clientHeight * window.devicePixelRatio;
camera.aspect = width / height;
camera.updateProjectionMatrix();
renderer.setSize(width, height, false); // YOU MUST PASS FALSE HERE!
pickingTexture.setSize(width, height);
}
function animate() {
requestAnimationFrame(animate);
controls.update();
render();
}
function render() {
pick();
renderer.render(scene, camera);
}
function pick() {
renderer.setRenderTarget(pickingTexture);
renderer.setClearColor(0);
renderer.render(scene, camera);
renderer.setClearColor(0xFFFFFF);
renderer.setRenderTarget(null)
//create buffer for reading single pixel
var pixelBuffer = new Uint8Array(4);
//read the pixel under the mouse from the texture
renderer.readRenderTargetPixels(pickingTexture, mouse.x * window.devicePixelRatio, pickingTexture.height - mouse.y * window.devicePixelRatio, 1, 1, pixelBuffer);
//interpret the pixel as an ID
var id = ( pixelBuffer[0] << 16 ) | ( pixelBuffer[1] << 8 ) | ( pixelBuffer[2] );
//if (id > 0) console.log(id);
info.textContent = id;
}
body {
color: #ffffff;
background-color: #000000;
margin: 0;
}
canvas { width: 100vw; height: 100vh; display: block; }
#info { position: absolute; left: 0; top: 0; color: red; background: black; padding: 0.5em; font-family: monospace; }
<script src="https://threejs.org/build/three.min.js"></script>
<script src="https://threejs.org/examples/js/controls/OrbitControls.js"></script>
<script type="x-shader/x-fragment" id="fragmentshader">
uniform sampler2D texture;
varying vec3 vColor;
void main() {
// solid squares of color
gl_FragColor = vec4( vColor, 1.0 );
}
</script>
<script type="x-shader/x-vertex" id="vertexshader">
attribute float size;
attribute vec3 customColor;
varying vec3 vColor;
void main() {
vColor = customColor;
vec4 mvPosition = modelViewMatrix * vec4( position, 1.0 );
gl_PointSize = size * ( 300.0 / length( mvPosition.xyz ) );
gl_Position = projectionMatrix * mvPosition;
}
</script>
<canvas id="c"></canvas>
<div id="info"></div>
请注意其他一些事情.
-
我猜您真的想将采摘纹理清除为零而不是白色.这样，0 =那里什么都没有，其他什么都没有.
I'd guess you really want to clear the picking texture to zero instead of white. That way 0 = nothing there, anything else = something there.
renderer.setClearColor(0);
renderer.render(scene, camera, pickingTexture);
renderer.setClearColor(0xFFFFFF);
不知道id <= numOfVertices的含义
因此，鉴于现在它已清除为零，因此代码只是
So given that it's clearing to zero now the code is just
if (id) console.log(id);
在初始化时，我没有设置渲染器大小，pickingTexture大小或相机外观.
I don't set the renderer size, the pickingTexture size nor the camera aspect at init time.
为什么要重复我自己. onWindowResize已经设置
Why repeat myself. onWindowResize already sets it
在调整画布大小以使其大小匹配时，您需要调整pickingTexture渲染目标的大小.
You need to resize the pickingTexture render target when the canvas is resizes so it matches in size.
我删除了对window.innerWidth和window.innerHeight
我会删除所有的代码，但是我不想为该示例更改更多代码.使用window.innerWidth将代码绑定到窗口.例如，如果您想在非全尺寸窗口中使用代码，则可以说您创建了编辑器.您必须更改代码.
I would have removed all of them but I didn't want to change even more code for this example. Using window.innerWidth ties the code to the window. If you ever want to use the code in something that's not the fullsize of the window, for example lets say you make an editor. You'll have to change the code.
以一种可以在更多情况下工作的方式编写代码并不难，所以为什么以后要自己做更多的工作.
It's not any harder to write the code in a way that works in more situations so why make more work for yourself later.
我没有选择的其他解决方案
Other solutions I didn't chose
-
您可以调用
render.setPixelRatio，然后使用window.devicePixelRatio
我没有选择此解决方案，因为您必须猜测three.js在后台做了什么.您的猜测今天可能是正确的，但明天可能是错误的.如果您告诉three.js 做点width by height 似乎更好，它应该只是做为width by height而不要做其他的事情.同样，您必须猜测three.js何时应用pixelRatio，何时不应用.正如您在上面注意到的那样，它不会将其应用于渲染目标的大小，并且因为它不知道目标是什么而无法应用.您是否正在将渲染目标作为拾取对象?要获得全屏效果?为了捕获?非全屏效果?由于无法知道，因此无法为您应用pixelRatio.这发生在Three.js代码中.它在某些地方应用pixelRatio，而其他地方则不应用.您还可以猜测.如果您从未设置pixelRatio，该问题将消失.
I didn't pick this solution because you have to guess what three.js is doing behind the scenes. Your guess might be correct today but wrong tomorrow. It seems better if you tell three.js make something width by height it should just make it width by height and not make it something else. Similarly you'd have to guess when three.js is going to apply pixelRatio and when it's not. As you noticed above it doesn't apply it to the size of the render target and it can't because it doesn't know what your purpose is. Are you making a render target for picking? For a fullscreen effect? For capture? for a non-fullscreen effect? Since it can't know it can't apply the pixelRatio for you. This happens all over the three.js code. Some places it applies pixelRatio, other places it doesn't. You're left guessing. If you never set pixelRatio that problem disappears.
您可以将devicePixelRatio传递到着色器中
You could pass in devicePixelRatio into your shader
<script type="x-shader/x-vertex" id="vertexshader">
attribute float size;
attribute vec3 customColor;
varying vec3 vColor;
uniform float devicePixelRatio; // added
void main() {
vColor = customColor;
vec4 mvPosition = modelViewMatrix * vec4( position, 1.0 );
gl_PointSize = size * ( 300.0 / length( mvPosition.xyz ) ) * devicePixelRatio;
gl_Position = projectionMatrix * mvPosition;
}
</script>
当然，您需要在制服中设置devicePixelRatio.
and of course you'd need to set devicePixelRatio in your uniforms.
我可能选择此解决方案.较小的问题是，如果pickingTexture的分辨率与画布的后缓冲区不同，则可能会出现1个错误.在这种情况下，如果画布是pickingTexture的2倍，则pickingTexture中不存在画布中每4个像素中的3个像素.取决于您的应用程序，但是可能没问题.您不能选择1/2像素，至少不能用鼠标来选择.
I might pick this solution. The minor problem is if the pickingTexture is not the same resolution as the canvas's backbuffer you can get off by 1 errors. In this case if the canvas was 2x the pickingTexture then 3 of every 4 pixels in the canvas don't exist in the pickingTexture. Depending on your application that might be ok though. You can't pick 1/2 pixels, at least not with the mouse.
我可能不会选择此解决方案的另一个其他原因是，它使问题弹出了其他地方. lineWidth是一个，gl_FragCoord是另一个.视口和剪刀设置也是如此.使渲染目标尺寸与画布相匹配似乎更好，以便使所有内容都相同，而不是进行越来越多的变通办法，并且必须记住在哪里使用一种尺寸与另一种尺寸.明天我开始使用PointsMaterial.它也与devicePixelRatio有关.通过不致电renderer.setPixelRatio，这些问题就消除了.
Another other reason I would probably not pick this solution is it just leaves the issue to pop up other places. lineWidth is one, gl_FragCoord is another. So are the viewport and scissor settings. It seems better to make the render target size match that canvas so that everything is the same rather than make more and more workarounds and have to remember where to use one size vs another. Tomorrow I start using the PointsMaterial. It also has issues with devicePixelRatio. By not calling renderer.setPixelRatio those problems go away.
这篇关于GPU拾取-子画面周围的不可见像素的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
