gpu 拾取 - 精灵周围的不可见像素 [英] gpu picking - invisible pixels around sprites
问题描述
我正在渲染一个包含精灵的采摘场景.当我的光标靠近精灵时，它会注册为一种颜色并被选中".当您放大精灵时，这个不可见的边框会变大.
I'm rendering a picking scene that contains sprites. As my cursor gets close to the sprite, it registers as a color and gets "picked". This invisible border gets larger closer you zoom into the sprites.
打开您的控制台以查看实时打印的 ID.将光标移近和远离大小精灵.您会看到精灵在不可见的边框上被选中.这种行为不会发生在常规几何体上，只会发生在精灵身上.
Open up your console to see the IDs printed in real time. Move your cursor closer and further away to large and small sprites. You'll see that sprites get selected on an invisible border. This behavior does not happen with regular geometry, just with sprites.
这很奇怪，因为我正在渲染 renderer.readRenderTargetPixels 实际看到的内容.
It's weird because I'm rendering out what renderer.readRenderTargetPixels actually sees.
如何去除不可见的边框以实现更准确的拣选?
How can I get rid of the invisible borders for more accurate picking?
var renderer, scene, camera, controls;
var particles, uniforms;
var PARTICLE_SIZE = 50;
var raycaster, intersects;
var mouse, INTERSECTED;
var pickingTexture;
var numOfVertices;
init();
animate();
function init() {
container = document.getElementById('container');
scene = new THREE.Scene();
camera = new THREE.PerspectiveCamera(45, window.innerWidth / window.innerHeight, 1, 10000);
camera.position.z = 150;
//
var geometry1 = new THREE.BoxGeometry(200, 200, 200, 4, 4, 4);
var vertices = geometry1.vertices;
numOfVertices = vertices.length;
var positions = new Float32Array(vertices.length * 3);
var colors = new Float32Array(vertices.length * 3);
var sizes = new Float32Array(vertices.length);
var vertex;
var color = new THREE.Color();
for (var i = 0, l = vertices.length; i < l; i++) {
vertex = vertices[i];
vertex.toArray(positions, i * 3);
color.setHex(i + 1);
color.toArray(colors, i * 3);
sizes[i] = PARTICLE_SIZE * 0.5;
}
var geometry = new THREE.BufferGeometry();
geometry.addAttribute('position', new THREE.BufferAttribute(positions, 3));
geometry.addAttribute('customColor', new THREE.BufferAttribute(colors, 3));
geometry.addAttribute('size', new THREE.BufferAttribute(sizes, 1));
//
var material = new THREE.ShaderMaterial({
uniforms: {
// texture: {type: "t", value: THREE.ImageUtils.loadTexture("../textures/circle.png")}
texture: {type: "t", value: THREE.ImageUtils.loadTexture("../textures/disc.png")}
},
vertexShader: document.getElementById('vertexshader').textContent,
fragmentShader: document.getElementById('fragmentshader').textContent,
depthTest: false,
transparent: false
// alphaTest: 0.9
});
//
particles = new THREE.Points(geometry, material);
scene.add(particles);
//
renderer = new THREE.WebGLRenderer({
antialias: true,
alpha: true
});
renderer.setPixelRatio(window.devicePixelRatio);
renderer.setSize(window.innerWidth, window.innerHeight);
renderer.setClearColor(0xffffff);
container.appendChild(renderer.domElement);
//
raycaster = new THREE.Raycaster();
mouse = new THREE.Vector2();
//
//
window.addEventListener('resize', onWindowResize, false);
document.addEventListener('mousemove', onDocumentMouseMove, false);
// defaults are on the right (except minFilter)
var options = {
format: THREE.RGBAFormat, // THREE.RGBAFormat
type: THREE.UnsignedByteType, // THREE.UnsignedByteType
anisotropy: 1, // 1
magFilter: THREE.LinearFilter, // THREE.LinearFilter
minFilter: THREE.LinearFilter, // THREE.LinearFilter
depthBuffer: true, // true
stencilBuffer: true // true
};
pickingTexture = new THREE.WebGLRenderTarget(window.innerWidth, window.innerHeight, options);
pickingTexture.texture.generateMipmaps = false;
controls = new THREE.OrbitControls(camera, container);
controls.damping = 0.2;
controls.enableDamping = false;
}
function onDocumentMouseMove(e) {
// event.preventDefault();
mouse.x = e.clientX;
mouse.y = e.clientY;
}
function onWindowResize() {
camera.aspect = window.innerWidth / window.innerHeight;
camera.updateProjectionMatrix();
renderer.setSize(window.innerWidth, window.innerHeight);
}
function animate() {
requestAnimationFrame(animate);
controls.update();
render();
}
function render() {
pick();
renderer.render(scene, camera);
}
function pick() {
renderer.render(scene, camera, pickingTexture);
//create buffer for reading single pixel
var pixelBuffer = new Uint8Array(4);
//read the pixel under the mouse from the texture
renderer.readRenderTargetPixels(pickingTexture, mouse.x, pickingTexture.height - mouse.y, 1, 1, pixelBuffer);
//interpret the pixel as an ID
var id = ( pixelBuffer[0] << 16 ) | ( pixelBuffer[1] << 8 ) | ( pixelBuffer[2] );
if (id <= numOfVertices) console.log(id);
}body {
color: #ffffff;
background-color: #000000;
margin: 0px;
overflow: hidden;
}<script src="http://threejs.org/build/three.min.js"></script>
<script src="http://threejs.org/examples/js/controls/OrbitControls.js"></script>
<script type="x-shader/x-fragment" id="fragmentshader">
uniform sampler2D texture;
varying vec3 vColor;
void main() {
// solid squares of color
gl_FragColor = vec4( vColor, 1.0 );
}
</script>
<script type="x-shader/x-vertex" id="vertexshader">
attribute float size;
attribute vec3 customColor;
varying vec3 vColor;
void main() {
vColor = customColor;
vec4 mvPosition = modelViewMatrix * vec4( position, 1.0 );
gl_PointSize = size * ( 300.0 / length( mvPosition.xyz ) );
gl_Position = projectionMatrix * mvPosition;
}
</script>
<div id="container"></div>推荐答案
问题是您使用的设备的 devicePixelRatio != 1.0 和three.js 的大小差不多.
The problem is you're on a device that has a devicePixelRatio != 1.0 and three.js lying about the size.
因为你调用了 renderer.setPixelRatio 现在魔法发生在幕后.您的画布不是您要求的尺寸，而是基于隐藏在 Three.js 代码中的某些公式的其他尺寸.
Because you called renderer.setPixelRatio now magic happens behind the scenes. Your canvas is not the size you requested it's some other size based on some formula hidden in the three.js code.
那么，会发生什么.您的画布是一种尺寸，但您的渲染目标是不同的尺寸.您的着色器使用 gl_PointSize 绘制其点.该大小以设备像素为单位.由于渲染目标的大小不同，渲染目标中的点大小与屏幕上的点大小不同.
So, what happens. Your canvas is one size but your render target is a different size. Your shader uses gl_PointSize to draw its points. That size is in device pixels. Because your render target is a different size the size of the points are different in your render target than they are on screen.
删除对 render.setPixelRatio 的调用，它将开始工作.
Remove the call to render.setPixelRatio and it will start working.
IMO 解决此问题的正确方法是自己使用 devicePixelRatio，因为这样发生的一切对您来说都是 100% 可见的.幕后没有魔法发生.
IMO the correct way to fix this is to use devicePixelRatio yourself because that way everything that is happening is 100% visible to you. No magic happening behind the scenes.
所以，
摆脱容器，直接使用画布
Get rid of the container and use a canvas directly
<canvas id="c"></canvas>
将画布设置为宽度使用 100vw，高度使用 100vh 并使主体 margin: 0;
set the canvas to use 100vw for width, 100vh for height and made the body margin: 0;
canvas { width: 100vw; height: 100vh; display: block; }
body { margin: 0; }
这将使您的画布自动拉伸以填充窗口.
This will make your canvas stretch automatically to fill the window.
使用浏览器拉伸画布的大小来选择它的drawingBuffer应该是的大小并乘以devicePixelRatio.假设您实际上想要支持设备像素比.无需执行两次此操作，因此请遵循 D.R.Y.，因此只需在 onWindowResize 中执行此操作.
Use the size the browser stretched the canvas to choose the size its drawingBuffer should be and multiply by devicePixelRatio. That assumes you actually want to support device pixel ratio. No need to do this twice so following D.R.Y. so just do it in onWindowResize.
canvas = document.getElementById("c");
renderer = new THREE.WebGLRenderer({
antialias: true,
alpha: true,
canvas: canvas,
});
pickingTexture = new THREE.WebGLRenderTarget(1, 1, options);
onWindowResize();
...
function onWindowResize() {
var width = canvas.clientWidth * window.devicePixelRatio;
var height = canvas.clientHeight * window.devicePixelRatio;
camera.aspect = width / height;
camera.updateProjectionMatrix();
renderer.setSize(width, height, false); // YOU MUST PASS FALSE HERE otherwise three.js will muck with the CSS
pickingTexture.setSize(width, height);
}
将鼠标坐标转换为设备坐标
Convert the mouse coordinates into device coordinates
renderer.readRenderTargetPixels(
pickingTexture,
mouse.x * window.devicePixelRatio,
pickingTexture.height - mouse.y * window.devicePixelRatio,
1, 1, pixelBuffer);
这是解决方案
var renderer, scene, camera, controls;
var particles, uniforms;
var PARTICLE_SIZE = 50;
var raycaster, intersects;
var mouse, INTERSECTED;
var pickingTexture;
var numOfVertices;
var info = document.querySelector('#info');
init();
animate();
function init() {
canvas = document.getElementById('c');
scene = new THREE.Scene();
camera = new THREE.PerspectiveCamera(45, 1, 1, 10000);
camera.position.z = 150;
//
var geometry1 = new THREE.BoxGeometry(200, 200, 200, 4, 4, 4);
var vertices = geometry1.vertices;
numOfVertices = vertices.length;
var positions = new Float32Array(vertices.length * 3);
var colors = new Float32Array(vertices.length * 3);
var sizes = new Float32Array(vertices.length);
var vertex;
var color = new THREE.Color();
for (var i = 0, l = vertices.length; i < l; i++) {
vertex = vertices[i];
vertex.toArray(positions, i * 3);
color.setHex(i + 1);
color.toArray(colors, i * 3);
sizes[i] = PARTICLE_SIZE * 0.5;
}
var geometry = new THREE.BufferGeometry();
geometry.setAttribute('position', new THREE.BufferAttribute(positions, 3));
geometry.setAttribute('customColor', new THREE.BufferAttribute(colors, 3));
geometry.setAttribute('size', new THREE.BufferAttribute(sizes, 1));
//
var loader = new THREE.TextureLoader();
var material = new THREE.ShaderMaterial({
uniforms: {
// texture: {type: "t", value: THREE.ImageUtils.loadTexture("../textures/circle.png")}
texture: {value: loader.load("https://i.imgur.com/iXT97XR.png")}
},
vertexShader: document.getElementById('vertexshader').textContent,
fragmentShader: document.getElementById('fragmentshader').textContent,
depthTest: false,
transparent: false
// alphaTest: 0.9
});
//
particles = new THREE.Points(geometry, material);
scene.add(particles);
//
renderer = new THREE.WebGLRenderer({
antialias: true,
alpha: true,
canvas: canvas,
});
renderer.setClearColor(0xffffff);
//
raycaster = new THREE.Raycaster();
mouse = new THREE.Vector2();
//
//
window.addEventListener('resize', onWindowResize, false);
document.addEventListener('mousemove', onDocumentMouseMove, false);
// defaults are on the right (except minFilter)
var options = {
format: THREE.RGBAFormat, // THREE.RGBAFormat
type: THREE.UnsignedByteType, // THREE.UnsignedByteType
anisotropy: 1, // 1
magFilter: THREE.LinearFilter, // THREE.LinearFilter
minFilter: THREE.LinearFilter, // THREE.LinearFilter
depthBuffer: true, // true
stencilBuffer: true // true
};
pickingTexture = new THREE.WebGLRenderTarget(1, 1, options);
pickingTexture.texture.generateMipmaps = false;
controls = new THREE.OrbitControls(camera, canvas);
controls.damping = 0.2;
controls.enableDamping = false;
onWindowResize();
}
function onDocumentMouseMove(e) {
// event.preventDefault();
mouse.x = e.clientX;
mouse.y = e.clientY;
}
function onWindowResize() {
var width = canvas.clientWidth * window.devicePixelRatio;
var height = canvas.clientHeight * window.devicePixelRatio;
camera.aspect = width / height;
camera.updateProjectionMatrix();
renderer.setSize(width, height, false); // YOU MUST PASS FALSE HERE!
pickingTexture.setSize(width, height);
}
function animate() {
requestAnimationFrame(animate);
controls.update();
render();
}
function render() {
pick();
renderer.render(scene, camera);
}
function pick() {
renderer.setRenderTarget(pickingTexture);
renderer.setClearColor(0);
renderer.render(scene, camera);
renderer.setClearColor(0xFFFFFF);
renderer.setRenderTarget(null)
//create buffer for reading single pixel
var pixelBuffer = new Uint8Array(4);
//read the pixel under the mouse from the texture
renderer.readRenderTargetPixels(pickingTexture, mouse.x * window.devicePixelRatio, pickingTexture.height - mouse.y * window.devicePixelRatio, 1, 1, pixelBuffer);
//interpret the pixel as an ID
var id = ( pixelBuffer[0] << 16 ) | ( pixelBuffer[1] << 8 ) | ( pixelBuffer[2] );
//if (id > 0) console.log(id);
info.textContent = id;
}body {
color: #ffffff;
background-color: #000000;
margin: 0;
}
canvas { width: 100vw; height: 100vh; display: block; }
#info { position: absolute; left: 0; top: 0; color: red; background: black; padding: 0.5em; font-family: monospace; }<script src="https://threejs.org/build/three.min.js"></script>
<script src="https://threejs.org/examples/js/controls/OrbitControls.js"></script>
<script type="x-shader/x-fragment" id="fragmentshader">
uniform sampler2D texture;
varying vec3 vColor;
void main() {
// solid squares of color
gl_FragColor = vec4( vColor, 1.0 );
}
</script>
<script type="x-shader/x-vertex" id="vertexshader">
attribute float size;
attribute vec3 customColor;
varying vec3 vColor;
void main() {
vColor = customColor;
vec4 mvPosition = modelViewMatrix * vec4( position, 1.0 );
gl_PointSize = size * ( 300.0 / length( mvPosition.xyz ) );
gl_Position = projectionMatrix * mvPosition;
}
</script>
<canvas id="c"></canvas>
<div id="info"></div>注意其他一些事情.
我猜你真的想将拾取纹理清除为零而不是白色.这样 0 = 什么都没有，其他什么 = 有什么.
I'd guess you really want to clear the picking texture to zero instead of white. That way 0 = nothing there, anything else = something there.
renderer.setClearColor(0);
renderer.render(scene, camera, pickingTexture);
renderer.setClearColor(0xFFFFFF);
不知道 id <= numOfVertices 是什么意思
鉴于现在清零，代码只是
So given that it's clearing to zero now the code is just
if (id) console.log(id);
我没有在初始化时设置渲染器大小、pickingTexture 大小和相机方面.
I don't set the renderer size, the pickingTexture size nor the camera aspect at init time.
为什么要重复我自己.onWindowResize 已经设置了
Why repeat myself. onWindowResize already sets it
您需要在调整画布大小时调整pickingTexture 渲染目标的大小，使其大小匹配.
You need to resize the pickingTexture render target when the canvas is resizes so it matches in size.
我删除了对 window.innerWidth 和 window.innerHeight
我会删除所有这些，但我不想为此示例更改更多代码.使用 window.innerWidth 将代码绑定到窗口.如果您想在不是窗口全尺寸的地方使用代码，例如假设您制作 一个编辑器.您必须更改代码.
I would have removed all of them but I didn't want to change even more code for this example. Using window.innerWidth ties the code to the window. If you ever want to use the code in something that's not the fullsize of the window, for example lets say you make an editor. You'll have to change the code.
以一种适用于更多情况的方式编写代码并不难，所以为什么以后要为自己做更多的工作.
It's not any harder to write the code in a way that works in more situations so why make more work for yourself later.
我没有选择的其他解决方案
Other solutions I didn't chose
您可以调用
render.setPixelRatio，然后使用window.devicePixelRatio
我没有选择这个解决方案是因为你必须猜测three.js在幕后做了什么.你的猜测今天可能是正确的，但明天可能是错误的.如果你告诉three.js 制作width by height它应该只是让它width by height而不是其他东西.同样，您必须猜测three.js 何时将应用pixelRatio，何时不应用.正如您在上面注意到的，它不会将它应用于渲染目标的大小，也不能，因为它不知道您的目的是什么.您是否正在制作用于拾取的渲染目标?为了全屏效果?为了捕捉?非全屏效果?因为它不知道它不能为您应用 pixelRatio.这发生在三个.js 代码中.有些地方它应用了 pixelRatio，其他地方则没有.你只能猜测了.如果您从不设置 pixelRatio，该问题就会消失.
I didn't pick this solution because you have to guess what three.js is doing behind the scenes. Your guess might be correct today but wrong tomorrow. It seems better if you tell three.js make something width by height it should just make it width by height and not make it something else. Similarly you'd have to guess when three.js is going to apply pixelRatio and when it's not. As you noticed above it doesn't apply it to the size of the render target and it can't because it doesn't know what your purpose is. Are you making a render target for picking? For a fullscreen effect? For capture? for a non-fullscreen effect? Since it can't know it can't apply the pixelRatio for you. This happens all over the three.js code. Some places it applies pixelRatio, other places it doesn't. You're left guessing. If you never set pixelRatio that problem disappears.
您可以将 devicePixelRatio 传入您的着色器
You could pass in devicePixelRatio into your shader
<script type="x-shader/x-vertex" id="vertexshader">
attribute float size;
attribute vec3 customColor;
varying vec3 vColor;
uniform float devicePixelRatio; // added
void main() {
vColor = customColor;
vec4 mvPosition = modelViewMatrix * vec4( position, 1.0 );
gl_PointSize = size * ( 300.0 / length( mvPosition.xyz ) ) * devicePixelRatio;
gl_Position = projectionMatrix * mvPosition;
}
</script>
当然，您还需要在制服中设置 devicePixelRatio.
and of course you'd need to set devicePixelRatio in your uniforms.
我可能选择这个解决方案.小问题是，如果 pickingTexture 与画布的后台缓冲区的分辨率不同，您可以通过 1 个错误下车.在这种情况下，如果画布是 pickingTexture 的 2 倍，那么画布中每 4 个像素中的 3 个像素不存在于 pickingTexture 中.根据您的应用程序，这可能没问题.你不能选择 1/2 像素，至少不能用鼠标.
I might pick this solution. The minor problem is if the pickingTexture is not the same resolution as the canvas's backbuffer you can get off by 1 errors. In this case if the canvas was 2x the pickingTexture then 3 of every 4 pixels in the canvas don't exist in the pickingTexture. Depending on your application that might be ok though. You can't pick 1/2 pixels, at least not with the mouse.
我可能不会选择这个解决方案的另一个原因是它只会让问题出现在其他地方.lineWidth 是一个，gl_FragCoord 是另一个.视口和剪刀设置也是如此.使渲染目标尺寸与画布匹配似乎更好，这样一切都相同，而不是做出越来越多的变通方法，并且必须记住在何处使用一种尺寸与另一种尺寸.明天我开始使用 PointsMaterial.它还存在 devicePixelRatio 问题.通过不调用 renderer.setPixelRatio，这些问题就会消失.
Another other reason I would probably not pick this solution is it just leaves the issue to pop up other places. lineWidth is one, gl_FragCoord is another. So are the viewport and scissor settings. It seems better to make the render target size match that canvas so that everything is the same rather than make more and more workarounds and have to remember where to use one size vs another. Tomorrow I start using the PointsMaterial. It also has issues with devicePixelRatio. By not calling renderer.setPixelRatio those problems go away.
这篇关于gpu 拾取 - 精灵周围的不可见像素的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
