跨设备的GPU选择不一致 [英] GPU Picking inconsistent across devices
问题描述
我正在尝试使用从本文后半部分修改的代码通过Points实现GPU拾取
I’m trying to implement GPU picking with Points using code I modified from the latter half of this article https://threejsfundamentals.org/threejs/lessons/threejs-picking.html
对于我来说，在台式机上运行正常，但是我开始测试不同的浏览器和设备，但无法始终如一地运行.我制作了Codepen来说明 https://codepen.io/deklanw/pen/OJVVmEd?编辑者= 1111
It’s been working fine for me on desktop, but I started testing different browsers and devices and it doesn’t work consistently. I made a Codepen to illustrate https://codepen.io/deklanw/pen/OJVVmEd?editors=1111
body {
margin: 0;
}
#c {
width: 100vw;
height: 100vh;
display: block;
}
<canvas id="c"></canvas>
<script type="module">
// Three.js - Picking - RayCaster w/Transparency
// from https://threejsfundamentals.org/threejs/threejs-picking-gpu.html
import * as THREE from "https://threejsfundamentals.org/threejs/resources/threejs/r113/build/three.module.js";
function main() {
const canvas = document.querySelector("#c");
const renderer = new THREE.WebGLRenderer({ canvas });
const fov = 60;
const aspect = 2; // the canvas default
const near = 0.1;
const far = 200;
const camera = new THREE.PerspectiveCamera(fov, aspect, near, far);
camera.position.z = 30;
const scene = new THREE.Scene();
scene.background = new THREE.Color(0);
const pickingScene = new THREE.Scene();
pickingScene.background = new THREE.Color(0);
// put the camera on a pole (parent it to an object)
// so we can spin the pole to move the camera around the scene
const cameraPole = new THREE.Object3D();
scene.add(cameraPole);
cameraPole.add(camera);
function randomNormalizedColor() {
return Math.random();
}
function getRandomInt(n) {
return Math.floor(Math.random() * n);
}
function getCanvasRelativePosition(e) {
const rect = canvas.getBoundingClientRect();
return {
x: e.clientX - rect.left,
y: e.clientY - rect.top
};
}
const textureLoader = new THREE.TextureLoader();
const particleTexture =
"https://raw.githubusercontent.com/mrdoob/three.js/master/examples/textures/sprites/ball.png";
const vertexShader = `
attribute float size;
attribute vec3 customColor;
varying vec3 vColor;
void main() {
vColor = customColor;
vec4 mvPosition = modelViewMatrix * vec4( position, 1.0 );
gl_PointSize = size * ( 100.0 / length( mvPosition.xyz ) );
gl_Position = projectionMatrix * mvPosition;
}
`;
const fragmentShader = `
uniform sampler2D texture;
varying vec3 vColor;
void main() {
vec4 tColor = texture2D( texture, gl_PointCoord );
if (tColor.a < 0.5) discard;
gl_FragColor = mix( vec4( vColor.rgb, 1.0 ), tColor, 0.1 );
}
`;
const pickFragmentShader = `
uniform sampler2D texture;
varying vec3 vColor;
void main() {
vec4 tColor = texture2D( texture, gl_PointCoord );
if (tColor.a < 0.25) discard;
gl_FragColor = vec4( vColor.rgb, 1.0);
}
`;
const materialSettings = {
uniforms: {
texture: {
type: "t",
value: textureLoader.load(particleTexture)
}
},
vertexShader: vertexShader,
fragmentShader: fragmentShader,
blending: THREE.NormalBlending,
depthTest: true,
transparent: false
};
const createParticleMaterial = () => {
const material = new THREE.ShaderMaterial(materialSettings);
return material;
};
const createPickingMaterial = () => {
const material = new THREE.ShaderMaterial({
...materialSettings,
fragmentShader: pickFragmentShader,
blending: THREE.NormalBlending
});
return material;
};
const geometry = new THREE.BufferGeometry();
const pickingGeometry = new THREE.BufferGeometry();
const colors = [];
const sizes = [];
const pickingColors = [];
const pickingColor = new THREE.Color();
const positions = [];
for (let i = 0; i < 30; i++) {
colors[3 * i] = randomNormalizedColor();
colors[3 * i + 1] = randomNormalizedColor();
colors[3 * i + 2] = randomNormalizedColor();
const rgbPickingColor = pickingColor.setHex(i + 1);
pickingColors[3 * i] = rgbPickingColor.r;
pickingColors[3 * i + 1] = rgbPickingColor.g;
pickingColors[3 * i + 2] = rgbPickingColor.b;
sizes[i] = getRandomInt(20);
positions[3 * i] = getRandomInt(20);
positions[3 * i + 1] = getRandomInt(20);
positions[3 * i + 2] = getRandomInt(20);
}
geometry.setAttribute(
"position",
new THREE.Float32BufferAttribute(positions, 3)
);
geometry.setAttribute(
"customColor",
new THREE.Float32BufferAttribute(colors, 3)
);
geometry.setAttribute("size", new THREE.Float32BufferAttribute(sizes, 1));
geometry.computeBoundingBox();
const material = createParticleMaterial();
const points = new THREE.Points(geometry, material);
// setup geometry and material for GPU picking
pickingGeometry.setAttribute(
"position",
new THREE.Float32BufferAttribute(positions, 3)
);
pickingGeometry.setAttribute(
"customColor",
new THREE.Float32BufferAttribute(pickingColors, 3)
);
pickingGeometry.setAttribute(
"size",
new THREE.Float32BufferAttribute(sizes, 1)
);
pickingGeometry.computeBoundingBox();
const pickingMaterial = createPickingMaterial();
const pickingPoints = new THREE.Points(pickingGeometry, pickingMaterial);
scene.add(points);
pickingScene.add(pickingPoints);
function resizeRendererToDisplaySize(renderer) {
const canvas = renderer.domElement;
const width = canvas.clientWidth;
const height = canvas.clientHeight;
const needResize = canvas.width !== width || canvas.height !== height;
if (needResize) {
renderer.setSize(width, height, false);
}
return needResize;
}
class GPUPickHelper {
constructor() {
// create a 1x1 pixel render target
this.pickingTexture = new THREE.WebGLRenderTarget(1, 1);
this.pixelBuffer = new Uint8Array(4);
}
pick(cssPosition, pickingScene, camera) {
const { pickingTexture, pixelBuffer } = this;
// set the view offset to represent just a single pixel under the mouse
const pixelRatio = renderer.getPixelRatio();
camera.setViewOffset(
renderer.getContext().drawingBufferWidth, // full width
renderer.getContext().drawingBufferHeight, // full top
(cssPosition.x * pixelRatio) | 0, // rect x
(cssPosition.y * pixelRatio) | 0, // rect y
1, // rect width
1 // rect height
);
// render the scene
renderer.setRenderTarget(pickingTexture);
renderer.render(pickingScene, camera);
renderer.setRenderTarget(null);
// clear the view offset so rendering returns to normal
camera.clearViewOffset();
//read the pixel
renderer.readRenderTargetPixels(
pickingTexture,
0, // x
0, // y
1, // width
1, // height
pixelBuffer
);
const id =
(pixelBuffer[0] << 16) | (pixelBuffer[1] << 8) | pixelBuffer[2];
console.log(`You clicked sphere number ${id}`);
return id;
}
}
const pickHelper = new GPUPickHelper();
function render(time) {
time *= 0.001; // convert to seconds;
if (resizeRendererToDisplaySize(renderer)) {
const canvas = renderer.domElement;
camera.aspect = canvas.clientWidth / canvas.clientHeight;
camera.updateProjectionMatrix();
}
cameraPole.rotation.y = time * 0.1;
renderer.render(scene, camera);
requestAnimationFrame(render);
}
requestAnimationFrame(render);
function onClick(e) {
const pickPosition = getCanvasRelativePosition(e);
const pickedID = pickHelper.pick(pickPosition, pickingScene, camera);
}
function onTouch(e) {
const touch = e.touches[0];
const pickPosition = getCanvasRelativePosition(touch);
const pickedID = pickHelper.pick(pickPosition, pickingScene, camera);
}
window.addEventListener("mousedown", onClick);
window.addEventListener("touchstart", onTouch);
}
main();
</script>
如果在节点上单击(或点击)，则其ID将在控制台中弹出.在某些设备上，我只能得到0，就像选择背景一样.
If you click (or tap) on the nodes their IDs should pop up in the console. On some devices I’m just getting 0, as in picking the background.
有人知道为什么吗?
此外，如果在这种情况下有一种方法(通过ShaderMaterial通过可变大小的点进行点网格划分)，并且仍然可以使用一种更简便的方法进行提取，我很好奇
Also, if there’s a way to do picking in this case (Point mesh with variable size points via ShaderMaterial) with an easier method that’s still performant, I’m curious about how
我删除了1x1渲染目标优化，似乎已经解决了.现在，我想知道该优化会导致什么问题.
I removed the 1x1 render target optimization and it seems to have fixed it. Now I'd like to know what about that optimization causes the problem..
推荐答案
问题是您不能在设备上以这种方式使用Point.
the problem is you can't use Points this way across devices.
一个点是否在中心不在屏幕上时是否绘制是独立于设备的(OpenGL ES/WebGL规范说仍然应该绘制，而OpenGL规范说没有绘制.没有针对它的测试，因此每个驱动程序都是不同)，而要解决这些问题，WebGL解决方案将需要太多工作，因此无法解决. AFAIK Intel和NVidia确实绘制了它们.基于AMD和PowerVR的(iPhone)不会绘制它们.
Whether a point is drawn when its center is offscreen or not is device independent (the OpenGL ES / WebGL spec says it's still supposed to be drawn, the OpenGL spec says it's not. There are no tests for it so each driver is different) and it would be too much work for WebGL implentations to work around so they don't. AFAIK Intel and NVidia do draw them. AMD and PowerVR based (iPhone) do not draw them.
如果使圆圈变大并确保其脱离屏幕(并且可能需要使画布变小)，则可以看到此问题.在某些设备上，它们将平滑地移出屏幕，而在其他设备上，它们的中心一旦移出屏幕，它们就会消失(通常取决于点的大小和视口的大小)
You can see this problem if you make the circles large and you make sure they go offscreen (and you may need to make your canvas small). On some devices they will smoothly go offscreen, on other devices as soon as their center goes offscreen they will disappear (often depending on the size of the point and the size of the viewport)
这意味着您的示例在任何情况下都不会真正起作用，无论是否使用1x1像素渲染目标，仅使用1x1像素渲染目标，几乎所有圆圈的中心都在1x1像素区域之外，因此它们不会不能在某些设备上绘制.当您使渲染目标与画布的大小匹配时，大多数圆的中心都在内部，但是边缘仍然会出现拾取错误.
This means your example does not really work in either case, with or without the 1x1 pixel render target it's just that with the 1x1 pixel render target pretty much all of the circles have their center outside that 1x1 pixel area so they don't get drawn on some devices. When you make the render target match the size of the canvas then most of the circles' centers are inside but you'll still get picking errors at the edges.
要解决此问题，您将需要使用四边形而不是点来绘制点.有很多方法可以做到这一点.将每个四边形绘制为单独的网格或精灵，或者将所有四边形合并到另一个网格中，或者使用InstancedMesh，其中每个点都需要一个矩阵，或者编写自定义着色器来做点(请参见本文)
To solve this you'll need to draw your points using quads instead of points. There are many ways to do that. Draw each quad as a separate mesh or sprite, or merge all the quads into another mesh, or use InstancedMesh where you'll need a matrix per point, or write custom shaders to do points (see the last example on this article)
请注意，要点也有其他问题.默认情况下，它们不会相对于画布大小进行缩放(当然，您可以在着色器中修复此问题，并且three.js也具有此选项).它们还具有与设备无关的最大尺寸，根据规格，该尺寸可以低至1像素.它们对设备像素比率设置的响应不佳(尽管您也可以在代码中对其进行修复).由于所有这些原因，积分的用途有限.可以说，代码绘制的大圆圈超出了该限制.
Note that points have other issues too. By default they don't scale relative to the canvas size (of course you can fix this in your shader and three.js has this option as well). They also have a device independent maximum size which according to the spec can be as low as 1 pixel. They don't respond well to device pixel ratio settings (though you could fix that in code as well). For all those reasons points have a limited uses. The large circles the code is drawing is arguably beyond that limit.
这篇关于跨设备的GPU选择不一致的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
