ShaderToy如何将声音加载到纹理中 [英] How does ShaderToy load sounds into a texture

问题描述

我一直试图做同样的事情，使用three.js将音频/波形传递到着色器中。

I've been trying to do the same things shadertoy does for passing audio frequency/waveform into the shader with three.js.

https://www.shadertoy.com/view/Xds3Rr

在此示例中，IQ似乎将频率/波形音频数据放入图像中，然后将其作为着色器中的纹理进行采样。如何在Javascript中创建音频纹理？

In this example it seems that IQ is putting frequency/waveform audio data into an image and then sampling that as a texture in the shader. How would I create that audio texture in Javascript?

要清楚，我不需要帮助将纹理均匀加载到着色器中。我只是不知道如何从音频文件创建音频纹理。

To be clear I don't need help loading the texture uniform into the shader. I just don't know how to create the audio texture from an audio file.

var texture = new THREE.Texture();

shader.uniforms = {
     iChannel0:  { type: 't', value: texture }
};

我猜我需要以某种方式将音频数据放入纹理我只是不要知道如何做到这一点。

I'm guessing I'll need to somehow put audio data into the texture I just don't know how to do that.

推荐答案

您可以从Web Audio API获取音频数据来创建分析器节点

You can get audio data from the Web Audio API be creating an analyser node

const audioContext = new window.AudioContext();
const analyser = audioContext.createAnalyser();

然后创建一个缓冲区来接收数据

Then create a buffer to receive teh data

const numSamples = analyser.frequencyBinCount;
const audioData = new Uint8Array(numSamples);

然后在渲染循环中获取数据并将其放入纹理

Then in your render loop get the data and put it in a texture

analyser.getByteFrequencyData(audioData);
...
gl.texImage2D(gl.TEXTURE_2D, 0, gl.LUMINANCE, numSamples, 1, 0,
              gl.LUMINANCE, gl.UNSIGNED_BYTE, audioData);

或在three.js中使用 DataTexture

or in three.js use a DataTexture

这是短版本。较长的版本是音频需要在同一个域上，否则您将遇到CORS问题。要获取音频流的数据，例如< audio> 标记，您需要调用

That's the short version. The longer version is audio needs to be on the same domain or you'll run into CORS issues. To get data for an audio stream like an <audio> tag's you'd call

const source = audioContext.createMediaElementSource(audio);

目前在移动Chrome或移动版Safari中无法使用，而且Safari中存在漏洞。

That doesn't work in mobile Chrome nor mobile Safari at the moment and there are bugs in Safari.

这是一份工作样本

这篇关于ShaderToy如何将声音加载到纹理中的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！