缩放视频纹理与图像纹理混合 [英] Scaling Video texture mixed with image texture

问题描述

我尝试过搜索，这是一件微不足道的事情，但无法弄清楚，我看了很多答案，但从未用完整的工作示例回答我需要的内容.

我正在尝试在片段着色器中缩放视频纹理，并在背景中使用图像纹理.所以不能变换顶点.视频纹理需要调整为 200 x 200 并放置在顶角或底角.

它最终将使用 MediaStream 作为网络摄像头，具有透明背景效果.它与图像纹理顶部的 bodypix 一起使用，使用混合着色器，但需要缩放视频，使其不会拉伸到视口.

如果我尝试使用 vec2(200,200) 的 vec2，clamp 将拉伸其余颜色.重复将平铺它.这些都不是预期的结果.

视频渲染的 Jsfiddle 但需要调整它的大小.

https://jsfiddle.net/danrossi303/or3dk2q4/16/>

我正在使用的基本片段着色器.

 precision mediump float;统一的sampler2D背景；统一的sampler2D框架；统一浮动 texWidth;统一浮动 texHeight；无效主(无效){vec2 texCoord = gl_FragCoord.xy/vec2(texWidth,texHeight);vec4 texel0 = 纹理2D(背景，texCoord)；vec4 texel1 = 纹理2D(框架，texCoord)；gl_FragColor = mix(texel0, texel1, 1.);}

解决方案

您需要缩放视频的纹理坐标:

vec2 frameuv = texCoord * vec2(texWidth, texHeight)/vec2(200.0, 200.0);vec4 texel1 = texture2D(frame, frameuv);

如果 frameuv.x 或 frameuv.y 大于 1.0，则丢弃框架 (texel1)(请参阅 step):

float w = step(frameuv.x, 1.0) * step(frameuv.y, 1.0);gl_FragColor = mix(texel0, texel1, w);

完整的片段着色器:

precision mediump float;统一的sampler2D背景；统一的sampler2D框架；统一浮动 texWidth;统一浮动 texHeight；无效主(无效){vec2 texCoord = gl_FragCoord.xy/vec2(texWidth,texHeight);vec4 texel0 = 纹理2D(背景，texCoord)；vec2 frameuv = texCoord * vec2(texWidth, texHeight)/vec2(200.0, 200.0);vec4 texel1 = 纹理2D(帧，frameuv)；gl_FragColor = mix(texel0, texel1, step(frameuv.x, 1.0) * step(frameuv.y, 1.0));}

---

此外，加载背景纹理时还有一个错误.gl.bindTexture 将纹理对象绑定到当前纹理单元.在调用 gl.bindTexture 之前，必须使用 gl.activeTexture 设置纹理单元:

img.onload = () =>{gl.activeTexture(gl.TEXTURE0);//<---gl.bindTexture(gl.TEXTURE_2D, 纹理);initBackgroundTexture();};img.src = "https://videos.electroteque.org/textures/virtualbg.jpg";

I've tried searching and is a trivial thing to do but can't figure it out, I looked at many answers but never answered what I need with full working examples.

I am trying to scale the video texture in a fragment shader, with an image texture in the background. So can't transform vertex. The video texture needs to be resized to 200 x 200 and positioned in the top or bottom corner.

It will use MediaStream eventually for webcam, with transparent background effect. It works with bodypix on top of an image texture, using the mix shader but need to scale the video so it doesn't stretch to the viewport.

If I try a vec2 of vec2(200,200), clamp will stretch the rest of the colours. Repeat will tile it. Neither of these is the expected result.

Jsfiddle of video render but need to resize it.

https://jsfiddle.net/danrossi303/or3dk2q4/16/

The basic fragment shader I am working with.

  precision mediump float;

 uniform sampler2D background;
 uniform sampler2D frame;

uniform float texWidth;
uniform float texHeight;


void main(void) {
vec2 texCoord = gl_FragCoord.xy / vec2(texWidth,texHeight);

vec4 texel0 = texture2D(background, texCoord);
vec4 texel1 = texture2D(frame, texCoord);
 gl_FragColor = mix(texel0, texel1, 1.);
}

解决方案

You need to scale the texture coordinates for the video:

vec2 frameuv = texCoord * vec2(texWidth, texHeight) / vec2(200.0, 200.0);
vec4 texel1 = texture2D(frame, frameuv);

Discard the frame (texel1) if frameuv.x or frameuv.y is greater than 1.0 (see step):

float w = step(frameuv.x, 1.0) * step(frameuv.y, 1.0);
gl_FragColor = mix(texel0, texel1, w);

Complete fragment shader:

precision mediump float;
  
uniform sampler2D background;
uniform sampler2D frame;

uniform float texWidth;
uniform float texHeight;

void main(void) {
    vec2 texCoord = gl_FragCoord.xy / vec2(texWidth,texHeight);
   
    vec4 texel0 = texture2D(background, texCoord);
    vec2 frameuv = texCoord * vec2(texWidth, texHeight) / vec2(200.0, 200.0);
    vec4 texel1 = texture2D(frame, frameuv);
    gl_FragColor = mix(texel0, texel1, step(frameuv.x, 1.0) * step(frameuv.y, 1.0));
}

---

Additionally there is a bug when you load the background texture. gl.bindTexture binds a texture object to the current texture unit. Therfor the texture unit has to be set with gl.activeTexture, before calling gl.bindTexture:

img.onload = () => {
    gl.activeTexture(gl.TEXTURE0);            // <---
    gl.bindTexture(gl.TEXTURE_2D, texture);
    initBackgroundTexture();
};
img.src = "https://videos.electroteque.org/textures/virtualbg.jpg"; 

这篇关于缩放视频纹理与图像纹理混合的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！