如何生成HTML5视频音量水平图? [英] How to generate HTML5 video volume level chart?
问题描述
给出一个简单的30秒的网络视频:
Given a plain web video of say 30s:
<video src="my-video.mp4"></video>
如何生成其音量水平图?
How could I generate its volume level chart?
volume|
level| ******
| * * **
| * * * **
|** * *** *
| ** * * * *
+---------------*-*-----************------+--- time
0 30s
video is and quiet
loud here here
注意:
- 请使用纯JavaScript.没有图书馆.
推荐答案
有几种方法，具体取决于用法.
There are several ways to do this depending on what the usage is.
为了准确起见，您可以使用常规的体积和单位进行度量，例如 RMS ，LUFS/ LKFS (K加权，响度)，
For accuracy you could measure in conventional volumes and units such as RMS, LUFS/LKFS (K-weighted, loudness), dBFS (full-scale dB) and so forth.
简单的简单方法是仅绘制波形的峰值.您只会对正值感兴趣.要仅获取峰，您将检测到两个点之间的方向，并在方向从上到下变化(p0> p1)时记录第一个点.
The simple naive approach is to just plot the peaks of the waveform. You would be interested in the positive values only. To just get the peaks you would detect direction between two points and log the first point when the direction changes from upward to downwards (p0 > p1).
对于所有方法，您最终都可以应用某种形式的平滑，例如加权移动平均值(示例)或通用平滑去除RMS和dB等小峰值和变化的算法.您可以使用可以与bin平滑(每段平均值)结合使用的窗口大小.
For all approaches you can finally apply some form of smoothing such as weighted moving average (example) or a generic smoothing algorithm to remove small peaks and changes, in case of RMS, dB etc. you would use a window size which can be combined with bin-smoothing (an average per segment).
要进行绘图，您将获得当前样本的值，假设已将其标准化，然后将其绘制为按绘图区域高度缩放的画布上的线或点.
To plot you will obtain the value for the current sample, assume it to be normalized and draw it as line or point to canvas scaled by plot area height.
解决评论中的一些问题；这些只是我的头上要给出一些指示-
To address some of the questions in the comments; these are just off the top of my heads to give some pointers -
由于Web Audio API无法单独进行流传输，因此您必须将整个文件加载到内存中，并将音频轨道解码到缓冲区中.
Since Web Audio API cannot do streaming on its own you have to load the entire file into memory and decode the audio track into a buffer.
- 优点:可以工作(分析部分)，可以在最终准备好数据时进行快速分析，可以处理较小的文件，如果缓存的URL可以使用而无需重新下载 缺点:初始加载时间长/用户体验不好，可能的内存占用/对大文件不利，音频分离".从视频同步开始，强制URL * 的重用，如果没有大的和/或缓存，则必须重新下载/流式传输文件，这当前在某些浏览器/版本中引起问题(请参阅下面的示例).
- Pros: works (analysis part), fast analysis when data is eventually ready, works fine for smaller files, if cached the URL can be used without re-downloading
- Cons: long initial load time/bad UX, possible memory hog/not good for large files, audio is "detached" from video sync-wise, forces reuse of URL*, if large and/or cache is not in place the file will have to be downloaded again/streamed, currently causes issues in some browsers/versions (see example below).
* :始终可以选择将下载的视频作为blob存储在IndexedDB中(及其含义)，并使用带有该blob的Object-URL在video元素中进行流式传输(可能需要MSE正常工作，还没有尝试过自己.)
在流式传输时进行绘图:
Plotting while streaming:
- 优点:内存/资源便宜
- 缺点:在整个文件播放完毕之前，剧情无法完整显示，用户可能会跳过/跳转部分，可能无法完成
侧面加载低质量的单声道纯音频文件:
Side-loading a low-quality mono audio-only file:
- 优点:可以将音频独立于视频文件加载到内存中，从而可以很好地近似水平使用
- 缺点:可能会延迟视频的初始加载，可能无法在视频开始播放之前及时准备好，
服务器端绘图:
- 优点:可以在上传时进行绘图，可以在请求视频时存储原始图数据(作为元数据提供)，低带宽，在视频开始时已准备好数据(假设数据表示的是时间段内的平均值).li>
- 缺点:要求服务器上的基础结构能够分离，分析和生成绘图数据，具体取决于数据的存储方式，可能需要修改数据库.
我可能遗漏了一些东西或错过了一些要点，但这应该可以大致理解...
I've might left out or missed some points, but it should give the general idea...
此示例测量每个样本给定窗口大小的常规dB.窗口大小越大，结果越平滑，但也将花费更多时间进行计算.
This example measures conventional dB of a given window size per sample. The bigger the window size the smoother the result, but will also take more time to calculate.
请注意，为简单起见，在此示例中，像素位置决定了dB窗口范围.根据影响当前样本值的缓冲区大小，这可能会产生不均匀的间隙/重叠，但应能达到此处说明的目的.同样为了简单起见，我通过将dB读数除以40来缩放dB，这在这里是一个任意数字(ABS只是用于绘图以及我在深夜/清晨这样做的时候大脑的工作方式(?):))
Note that for simplicity in this example pixel position determines the dB window range. This may produce uneven gaps/overlaps depending on buffer size affecting the current sample value, but should work for the purpose demonstrated here. Also for simplicity I am scaling the dB reading by dividing it by 40, a somewhat arbitrary number here (ABS is just for the plotting and the way my brain worked (?) in the late night/early morning when I made this :) ).
我在顶部添加了红色的bin/segment-smoothing，以便更好地显示与自动调平等事情有关的长期音频变化.
I added bin/segment-smoothing in red on top to better show longer-term audio variations relevant to things such as auto-leveling.
我在这里使用音频源，但是您可以插入视频源，只要它包含可以解码的音轨格式(aac，mp3，ogg等)即可.
I'm using a audio source here but you can plug in a video source instead as long as it contains an audio track format that can be decoded (aac, mp3, ogg etc.).
除此之外，例子就是那个例子.这不是生产代码，因此请考虑它的价值.根据需要进行调整.
Besides from that, the example is just that, an example. It's not production code so take it for what it is worth. Make adjustments as needed.
(由于某些原因，音频无法在Firefox v58beta中播放，但会进行绘制.音频在Chrome，FF58dev中播放).
var ctx = c.getContext("2d"), ref, audio;
var actx = new (AudioContext || webkitAudioContext)();
var url = "//dl.dropboxusercontent.com/s/a6s1qq4lnwj46uj/testaudiobyk3n_lo.mp3";
ctx.font = "20px sans-serif";
ctx.fillText("Loading and processing...", 10, 50);
ctx.fillStyle = "#001730";
// Load audio
fetch(url, {mode: "cors"})
.then(function(resp) {return resp.arrayBuffer()})
.then(actx.decodeAudioData.bind(actx))
.then(function(buffer) {
// Get data from channel 0 (you will want to measure all/avg.)
var channel = buffer.getChannelData(0);
// dB per window + Plot
var points = [0];
ctx.clearRect(0, 0, c.width, c.height);
ctx.moveTo(x, c.height);
for(var x = 1, i, v; x < c.width; x++) {
i = ((x / c.width) * channel.length)|0; // get index in buffer based on x
v = Math.abs(dB(channel, i, 8820)) / 40; // 200ms window, normalize
ctx.lineTo(x, c.height * v);
points.push(v);
}
ctx.fill();
// smooth using bins
var bins = 40; // segments
var range = (c.width / bins)|0;
var sum;
ctx.beginPath();
ctx.moveTo(0,c.height);
for(x = 0, v; x < points.length; x++) {
for(v = 0, i = 0; i < range; i++) {
v += points[x++];
}
sum = v / range;
ctx.lineTo(x - (range>>1), sum * c.height); //-r/2 to compensate visually
}
ctx.lineWidth = 2;
ctx.strokeStyle = "#c00";
ctx.stroke();
// for audio / progressbar only
c.style.backgroundImage = "url(" + c.toDataURL() + ")";
c.width = c.width;
ctx.fillStyle = "#c00";
audio = document.querySelector("audio");
audio.onplay = start;
audio.onended = stop;
audio.style.display = "block";
});
// calculates RMS per window and returns dB
function dB(buffer, pos, winSize) {
for(var rms, sum = 0, v, i = pos - winSize; i <= pos; i++) {
v = i < 0 ? 0 : buffer[i];
sum += v * v;
}
rms = Math.sqrt(sum / winSize); // corrected!
return 20 * Math.log10(rms);
}
// for progress bar (audio)
function start() {if (!ref) ref = requestAnimationFrame(progress)}
function stop() {cancelAnimationFrame(ref);ref=null}
function progress() {
var x = audio.currentTime / audio.duration * c.width;
ctx.clearRect(0,0,c.width,c.height);
ctx.fillRect(x-1,0,2,c.height);
ref = requestAnimationFrame(progress)
}body {background:#536375}
#c {border:1px solid;background:#7b8ca0}<canvas id=c width=640 height=300></canvas><br>
<audio style="display:none" src="//dl.dropboxusercontent.com/s/a6s1qq4lnwj46uj/testaudiobyk3n_lo.mp3" controls></audio>这篇关于如何生成HTML5视频音量水平图?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
