高分辨率使用webgl绘制画布 [英] high resolution draw canvas with webgl

问题描述

我想获取我的webgl画布的快照，并且想要高分辨率捕获，所以我增加了画布的大小.这将自动更改gl.draingBufferWidth和gl.draingBufferWidth.然后设置视口，然后渲染场景.

I want to get a snapshot of my webgl canvas and I want a high resolution capture so I increased my canvas size. This automatically changes gl.draingBufferWidth and gl.draingBufferWidth. I then set viewport and then render the scene.

我的代码在低分辨率(4000 * 4000以下)下可以正常工作，但在高分辨率下存在很多问题.

My code works correctly in low resolution (under 4000*4000) but in higher resolutions there are many problems.

如果分辨率稍高，则快照不会完全显示.查看附件.如果分辨率提高更多，则不会显示任何内容.最后，在某些解决方案下，我的webgl实例被破坏了，我必须重新启动浏览器才能使webgl再次运行

If the resolution is a bit higher the snapshot not does not completely show. See attached file. If the resolution increases more nothing is shown. And finally at some resolutions my instance of webgl is destroyed and I have to restart the browser to get webgl running again

有什么方法可以从webgl画布获取高分辨率的快照吗?我可以使用其他解决方案吗?

Is there any way to get a snapshot from webgl canvas with a high-resolution ? Can I use another solution?

推荐答案

4000x4000像素是4000x4000x4或64兆像素的内存. 8000x8000是256兆的内存.浏览器不喜欢分配那么大的内存，并且经常在页面上设置限制.因此，例如，您有一个8000x8000 WebGL画布，它需要2个缓冲区.页面上显示的drawingbuffer和纹理.绘图缓冲区可能是抗锯齿.如果它是MSAA的4倍，那么仅该缓冲区就需要一个千兆内存.然后，您截取屏幕截图，以便再增加256Meg的内存.因此，是的，浏览器由于某种原因很可能会杀死您的页面.

4000x4000 pixel is 4000x4000x4 or 64meg of memory. 8000x8000 is 256meg of memory. Browser's don't like allocating that large chunks of memory and often set limits on the page. So for example you have an 8000x8000 WebGL canvas which requires 2 buffers. The drawingbuffer AND the texture being displayed on the page. The drawingbuffer might be anti-aliases. If it's 4x MSAA then it would require a gig of memory just for that buffer. Then you take a screenshot so another 256meg of memory. So yes, the browser for one reason or another is likely to kill your page.

最重要的是，WebGL有其自身的大小限制.您可以查找实际上是 MAX_TEXTURE_SIZE 或仅将其过滤到桌面上会更好).该数字仅表示硬件可以执行的操作.仍然受内存限制.

On top of that WebGL has it's own limits in size. You can look up that limit which is effectively MAX_TEXTURE_SIZE or MAX_VIEWPORT_DIMS. You can see from those about 40% of machines can't drawing larger than 4096 (although if you filter to desktop only it's much better). That number only means what the hardware can do. It's still limited by memory.

一种可能解决此问题的方法是将图像分成几部分绘制.如何执行将取决于您的应用程序.如果对所有渲染都使用相当标准的透视矩阵，则可以使用略有不同的数学来渲染视图的任何部分.大多数3d数学库都具有perspective函数，并且大多数还具有相应的frustum函数，该函数稍微灵活一些.

One way to kind of maybe solve this issue is to draw the image in parts. How you do that will depend on your app. If you're using a fairly standard perspective matrix for all your rendering you can use slightly different math to render any portion of the view. Most 3d math libraries have a perspective function and most of them also have a corresponding frustum function that is slightly more flexible.

这是一个相当标准的WebGL简单样式示例，它使用典型的perspective函数绘制一个立方体

Here's a fairly standard style WebGL simple sample that draws a cube using a typical perspective function

"use strict";

const vs = `
uniform mat4 u_worldViewProjection;

attribute vec4 position;
attribute vec3 normal;

varying vec3 v_normal;

void main() {
  v_normal = normal;
  gl_Position = u_worldViewProjection * position;
}
`;
const fs = `
precision mediump float;

varying vec3 v_normal;

void main() {
  gl_FragColor = vec4(v_normal * .5 + .5, 1);
}
`;

const m4 = twgl.m4;
const gl = document.querySelector("canvas").getContext("webgl");
const programInfo = twgl.createProgramInfo(gl, [vs, fs]);

const bufferInfo = twgl.primitives.createCubeBufferInfo(gl, 2);

twgl.resizeCanvasToDisplaySize(gl.canvas);
gl.viewport(0, 0, gl.canvas.width, gl.canvas.height);

gl.enable(gl.DEPTH_TEST);
gl.enable(gl.CULL_FACE);
gl.clearColor(0.2, 0.2, 0.2, 1);
gl.clear(gl.COLOR_BUFFER_BIT | gl.DEPTH_BUFFER_BIT);

const fov = 30 * Math.PI / 180;
const aspect = gl.canvas.clientWidth / gl.canvas.clientHeight;
const zNear = 0.5;
const zFar = 10;
const projection = m4.perspective(fov, aspect, zNear, zFar);
const eye = [1, 4, -6];
const target = [0, 0, 0];
const up = [0, 1, 0];

const camera = m4.lookAt(eye, target, up);
const view = m4.inverse(camera);
const viewProjection = m4.multiply(projection, view);
const world = m4.rotationY(Math.PI * .33);

gl.useProgram(programInfo.program);
twgl.setBuffersAndAttributes(gl, programInfo, bufferInfo);
twgl.setUniforms(programInfo, {
  u_worldViewProjection: m4.multiply(viewProjection, world),
});
twgl.drawBufferInfo(gl, bufferInfo);

<canvas></canvas>
<script src="https://twgljs.org/dist/4.x/twgl-full.min.js"></script>

这是使用典型的frustum函数而不是perspective

And here's the same code rendering at 400x200 in eight 100x100 parts using a typical frustum function instead of perspective

"use strict";

const vs = `
uniform mat4 u_worldViewProjection;

attribute vec4 position;
attribute vec3 normal;

varying vec3 v_normal;

void main() {
  v_normal = normal;
  gl_Position = u_worldViewProjection * position;
}
`;
const fs = `
precision mediump float;

varying vec3 v_normal;

void main() {
  gl_FragColor = vec4(v_normal * .5 + .5, 1);
}
`;

const m4 = twgl.m4;
const gl = document.createElement("canvas").getContext("webgl");
const programInfo = twgl.createProgramInfo(gl, [vs, fs]);

const bufferInfo = twgl.primitives.createCubeBufferInfo(gl, 2);

// size to render
const totalWidth = 400;
const totalHeight = 200;
const partWidth = 100;
const partHeight = 100;

// this fov is for the totalHeight
const fov = 30 * Math.PI / 180;
const aspect = totalWidth / totalHeight;
const zNear = 0.5;
const zFar = 10;

const eye = [1, 4, -6];
const target = [0, 0, 0];
const up = [0, 1, 0];

// since the camera doesn't change let's compute it just once
const camera = m4.lookAt(eye, target, up);
const view = m4.inverse(camera);
const world = m4.rotationY(Math.PI * .33);

const imgRows = []; // this is only to insert in order
for (let y = 0; y < totalHeight; y += partHeight) {
  const imgRow = [];
  imgRows.push(imgRow)
  for (let x = 0; x < totalWidth; x += partWidth) {
    renderPortion(totalWidth, totalHeight, x, y, partWidth, partHeight);
    const img = new Image();
    img.src = gl.canvas.toDataURL();
    imgRow.push(img);
  }
}

// because webgl goes positive up we're generating the rows
// bottom first
imgRows.reverse().forEach((imgRow) => {
  imgRow.forEach(document.body.appendChild.bind(document.body));
  document.body.appendChild(document.createElement("br"));
});

function renderPortion(totalWidth, totalHeight, partX, partY, partWidth, partHeight) {
  gl.canvas.width = partWidth;
  gl.canvas.height = partHeight;
  
  gl.viewport(0, 0, gl.canvas.width, gl.canvas.height);
  
  gl.enable(gl.DEPTH_TEST);
  gl.enable(gl.CULL_FACE);
  gl.clearColor(0.2, 0.2, 0.2, 1);
  gl.clear(gl.COLOR_BUFFER_BIT | gl.DEPTH_BUFFER_BIT);

  // corners at zNear for tital image
  const zNearTotalTop = Math.tan(fov) * 0.5 * zNear;
  const zNearTotalBottom = -zNearTotalTop;
  const zNearTotalLeft = zNearTotalBottom * aspect;
  const zNearTotalRight = zNearTotalTop * aspect;
  
  // width, height at zNear for total image
  const zNearTotalWidth = zNearTotalRight - zNearTotalLeft;
  const zNearTotalHeight = zNearTotalTop - zNearTotalBottom;
  
  const zNearPartLeft = zNearTotalLeft + partX * zNearTotalWidth / totalWidth;   const zNearPartRight = zNearTotalLeft + (partX + partWidth) * zNearTotalWidth / totalWidth;
  const zNearPartBottom = zNearTotalBottom + partY * zNearTotalHeight / totalHeight;
  const zNearPartTop = zNearTotalBottom + (partY + partHeight) * zNearTotalHeight / totalHeight;

  const projection = m4.frustum(zNearPartLeft, zNearPartRight, zNearPartBottom, zNearPartTop, zNear, zFar);
  const viewProjection = m4.multiply(projection, view);

  gl.useProgram(programInfo.program);
  twgl.setBuffersAndAttributes(gl, programInfo, bufferInfo);
  twgl.setUniforms(programInfo, {
    u_worldViewProjection: m4.multiply(viewProjection, world),
  });
  twgl.drawBufferInfo(gl, bufferInfo);
}

img { border: 1px solid red; }
body { line-height: 0 }

<script src="https://twgljs.org/dist/4.x/twgl-full.min.js"></script>

如果运行上面的代码段，您会看到它正在生成8张图像

If you run the snippet above you'll see it's generating 8 images

重要的部分是这个

首先，我们需要确定所需的总大小

First we need to decide on the total size we want

const totalWidth = 400;
const totalHeight = 200;

然后我们将创建一个函数，该函数将呈现该大小的任何较小部分

Then we'll make a function that will render any smaller portion of that size

function renderPortion(totalWidth, totalHeight, partX, partY, partWidth, partHeight) {
   ...

我们将画布设置为零件的大小

We'll set the canvas to the size of the part

  gl.canvas.width = partWidth;
  gl.canvas.height = partHeight;

  gl.viewport(0, 0, gl.canvas.width, gl.canvas.height);

，然后计算我们需要传递给frustum函数的内容.首先，我们在zNear处计算一个矩形，该矩形将根据给定的视场，宽高比和zNear值来构建透视矩阵.

And then compute what we need to pass to the frustum function. First we compute the rectangle at zNear that a perspective matrix would make given our field of view, aspect, and zNear values

  // corners at zNear for total image
  const zNearTotalTop = Math.tan(fov) * 0.5 * zNear;
  const zNearTotalBottom = -zNearTotalTop;
  const zNearTotalLeft = zNearTotalBottom * aspect;
  const zNearTotalRight = zNearTotalTop * aspect;

  // width, height at zNear for total image
  const zNearTotalWidth = zNearTotalRight - zNearTotalLeft;
  const zNearTotalHeight = zNearTotalTop - zNearTotalBottom;

然后，我们针对要渲染的部分在zNear处计算相应的面积，并将其传递给frustum以生成投影矩阵.

Then we compute the corresponding area at zNear for the part of that we want to render and pass those to frustum to generate a projection matrix.

  const zNearPartLeft = zNearTotalLeft + partX * zNearTotalWidth / totalWidth;   const zNearPartRight = zNearTotalLeft + (partX + partWidth) * zNearTotalWidth / totalWidth;
  const zNearPartBottom = zNearTotalBottom + partY * zNearTotalHeight / totalHeight;
  const zNearPartTop = zNearTotalBottom + (partY + partHeight) * zNearTotalHeight / totalHeight;

  const projection = m4.frustum(zNearPartLeft, zNearPartRight, zNearPartBottom, zNearPartTop, zNear, zFar);

然后我们就像正常渲染一样

Then we just render like normal

最后在外面，我们有一个循环，可以使用刚生成的函数以所需的任意分辨率渲染任意数量的零件.

Finally on the outside we have a loop to use the function we just generated to render as many parts as we want at whatever resolution we want.

const totalWidth = 400;
const totalHeight = 200;
const partWidth = 100;
const partHeight = 100;

for (let y = 0; y < totalHeight; y += partHeight) {
  for (let x = 0; x < totalWidth; x += partWidth) {
    renderPortion(totalWidth, totalHeight, x, y, partWidth, partHeight);
    const img = new Image();
    img.src = gl.canvas.toDataURL();
    // do something with image.
  }
}

这将使您可以渲染为所需的任何大小，但是还需要其他方法将图像组合成一个较大的图像.您可能会或可能不会在浏览器中执行此操作.您可以尝试制作一个巨大的2D画布并在其中绘制每个零件(假设2D画布没有与WebGL相同的限制).为此，无需制作图像，只需将webgl画布绘制到2d画布中即可.

This will let you render to any size you want but you'll need some other way to assemble the images into one larger image. You may or may not be able to do that in the browser. You could try making a giant 2D canvas and drawing each part into it (that assumes 2d canvas doesn't have the same limits as WebGL). To do that there's no need to make the images, just draw the webgl canvas into the 2d canvas.

否则，您可能必须将它们发送到您创建的服务器以组装图像，或者根据用例，让用户保存它们并将它们全部加载到图像编辑程序中.

Otherwise you might have to send them to a server you create to assemble the image or depending on your use case let the user save them and load them all into an image editing program.

或者，如果您只想显示它们，则浏览器使用16x16 1024x1024图像的效果可能会好于一张16kx16k图像.在这种情况下，您可能想调用canvas.toBlob而不是使用dataURL，然后为每个Blob调用URL.createObjectURL.这样一来，您就不会再出现这些巨大的dataURL字符串了.

Or if you just want to display them the browser will probably do better with 16x16 1024x1024 images than one 16kx16k image. In that case you probably want to call canvas.toBlob instead of using dataURLs and then call URL.createObjectURL for each blob. That way you won't have these giant dataURL strings sitting around.

示例:

"use strict";

const vs = `
uniform mat4 u_worldViewProjection;

attribute vec4 position;
attribute vec3 normal;

varying vec3 v_normal;

void main() {
  v_normal = normal;
  gl_Position = u_worldViewProjection * position;
}
`;
const fs = `
precision mediump float;

varying vec3 v_normal;

void main() {
  gl_FragColor = vec4(v_normal * .5 + .5, 1);
}
`;

const m4 = twgl.m4;
const gl = document.createElement("canvas").getContext("webgl");
const programInfo = twgl.createProgramInfo(gl, [vs, fs]);

const bufferInfo = twgl.primitives.createCubeBufferInfo(gl, 2);

// size to render
const totalWidth = 16384;
const totalHeight = 16385;
const partWidth = 1024;
const partHeight = 1024;

// this fov is for the totalHeight
const fov = 30 * Math.PI / 180;
const aspect = totalWidth / totalHeight;
const zNear = 0.5;
const zFar = 10;

const eye = [1, 4, -6];
const target = [0, 0, 0];
const up = [0, 1, 0];

// since the camera doesn't change let's compute it just once
const camera = m4.lookAt(eye, target, up);
const view = m4.inverse(camera);
const world = m4.rotationY(Math.PI * .33);

const imgRows = []; // this is only to insert in order
for (let y = 0; y < totalHeight; y += partHeight) {
  const imgRow = [];
  imgRows.push(imgRow)
  for (let x = 0; x < totalWidth; x += partWidth) {
    renderPortion(totalWidth, totalHeight, x, y, partWidth, partHeight);
    const img = new Image();
    gl.canvas.toBlob((blob) => {
      img.src = URL.createObjectURL(blob);
    });
    imgRow.push(img);
  }
}

// because webgl goes positive up we're generating the rows
// bottom first
imgRows.reverse().forEach((imgRow) => {
  const div = document.createElement('div');
  imgRow.forEach(div.appendChild.bind(div));
  document.body.appendChild(div);
});

function renderPortion(totalWidth, totalHeight, partX, partY, partWidth, partHeight) {
  gl.canvas.width = partWidth;
  gl.canvas.height = partHeight;
  
  gl.viewport(0, 0, gl.canvas.width, gl.canvas.height);
  
  gl.enable(gl.DEPTH_TEST);
  gl.enable(gl.CULL_FACE);
  gl.clearColor(0.2, 0.2, 0.2, 1);
  gl.clear(gl.COLOR_BUFFER_BIT | gl.DEPTH_BUFFER_BIT);

  // corners at zNear for tital image
  const zNearTotalTop = Math.tan(fov) * 0.5 * zNear;
  const zNearTotalBottom = -zNearTotalTop;
  const zNearTotalLeft = zNearTotalBottom * aspect;
  const zNearTotalRight = zNearTotalTop * aspect;
  
  // width, height at zNear for total image
  const zNearTotalWidth = zNearTotalRight - zNearTotalLeft;
  const zNearTotalHeight = zNearTotalTop - zNearTotalBottom;
  
  const zNearPartLeft = zNearTotalLeft + partX * zNearTotalWidth / totalWidth;   const zNearPartRight = zNearTotalLeft + (partX + partWidth) * zNearTotalWidth / totalWidth;
  const zNearPartBottom = zNearTotalBottom + partY * zNearTotalHeight / totalHeight;
  const zNearPartTop = zNearTotalBottom + (partY + partHeight) * zNearTotalHeight / totalHeight;

  const projection = m4.frustum(zNearPartLeft, zNearPartRight, zNearPartBottom, zNearPartTop, zNear, zFar);
  const viewProjection = m4.multiply(projection, view);

  gl.useProgram(programInfo.program);
  twgl.setBuffersAndAttributes(gl, programInfo, bufferInfo);
  twgl.setUniforms(programInfo, {
    u_worldViewProjection: m4.multiply(viewProjection, world),
  });
  twgl.drawBufferInfo(gl, bufferInfo);
}

img { border: 1px solid red; }
div { white-space: nowrap; }
body { line-height: 0 }

<script src="https://twgljs.org/dist/4.x/twgl-full.min.js"></script>

如果您希望用户能够下载16386x16386图像而不是256个1024x1024图像，那么另一种解决方案是使用上面的部件渲染代码，并对图像的每一行(或多行)将其数据写入blob手动生成PNG. 这篇博客文章介绍了如何根据数据手动生成PNG，并且此答案建议了如何针对非常大的数据进行处理.

If you want the user to be able to download a 16386x16386 image instead of 256 1024x1024 images then yet one more solution is to use the part rendering code above and for each row (or rows) of images write their data to a blobs to manually generate a PNG. This blog post covers manually generating PNGs from data and this answer suggests how to do it for very large data.

只是为了好玩，我写了这个库来帮助在浏览器中生成巨大的png .

Just for fun I wrote this library to help generate giant pngs in the browser.

这篇关于高分辨率使用webgl绘制画布的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！