检测 MediaStreamTrack 是否为黑色/空白 [英] Detect if MediaStreamTrack is black/blank

问题描述

我正在使用 peerjs 创建视频聊天.

I'm creating videochat with peerjs.

我正在使用以下功能切换相机(开/关):

I'm toggling camera (on/off) with the following function:

function toggleCamera() {
    localStream.getVideoTracks()[0].enabled = !(localStream.getVideoTracks()[0].enabled);
}

调用此函数后，视频变黑，接收器变黑(按预期工作).现在我想检测黑/白屏，以便向用户显示一些消息或图标，表明相机已禁用且没有流.

After calling this function, video goes black and receiver gets just black screen (which works as intended). Now I want to detect black/blank screen so I can show user some message or icon that camera is disabled and there is no stream.

我该如何检测?

推荐答案

一段时间后，我设法找到了解决方案:

After some time I've managed to get solution:

var previousBytes = 0;
var previousTS = 0;
var currentBytes = 0;
var currentTS = 0;

// peer - new Peer()
// stream - local camera stream (received from navigator.mediaDevices.getUserMedia(constraints))
let connection = peer.call(peerID, stream);

// peerConnection - reference to RTCPeerConnection (https://peerjs.com/docs.html#dataconnection-peerconnection)
connection.peerConnection.getStats(null).then(stats => {
    stats.forEach(report => {
        if (report.type === "inbound-rtp") {

            currentBytes = report.bytesReceived;
            currentTS = report.timestamp;

            if (previousBytes == 0) {
                previousBytes = currentBytes;
                previousTS = currentTS;
                return;
            }

            console.log({ previousBytes })
            console.log({ currentBytes })

            var deltaBytes = currentBytes - previousBytes;
            var deltaTS = currentTS - previousTS;

            console.log("Delta: " + (deltaBytes / deltaTS) + " kB/s")
            previousBytes = currentBytes;
            previousTS = currentTS;

        }
    });
});

这段代码实际上是在每秒被调用的函数中.当相机打开但未被覆盖时，deltaBytes 介于 100 和 250 之间，当相机关闭(以编程方式)或覆盖(用餐巾纸或其他东西)时，因此相机流为黑色/空白，deltaBytes 为 1.5-3kbps.重新打开相机后，deltaBytes 会出现峰值，达到 500kbps 左右.

This code is actually in function which gets called every second. When camera is turned on and it's not covered, deltaBytes is between 100 and 250, when camera is turned off (programmatically) or covered (with a napkin or something), so camera stream is black/blank, deltaBytes is med 1.5-3kbps. After you turn camera back on, there is a spike in deltaBytes, which reaches around 500kbps.

这是简短的控制台日志:

This is short console log:

    124.52747252747253 kB/s
    202.213 kB/s
    194.64764764764766 kB/s
    15.313 kB/s (this is where camera is covered)
    11.823823823823824 kB/s
    11.862137862137862 kB/s
    2.164 kB/s
    2.005 kB/s
    2.078078078078078 kB/s
    1.99 kB/s
    2.059 kB/s
    1.992992992992993 kB/s
    159.89810189810188 kB/s (uncovered camera)
    502.669 kB/s
    314.7927927927928 kB/s
    255.0909090909091 kB/s
    220.042 kB/s
    213.46353646353646 kB/s

所以最后我按照@Philipp Hancke 说的做了.我创建了从页面加载到用户关闭它的主连接.通过此连接，我正在发送用于启动视频通话、取消视频会话、打开/关闭摄像头的命令……然后在另一边我正在解析这些命令并执行函数.

So in the end I did as @Philipp Hancke said. I created master connection which is open from when the page loads until user closes it. Over this connection I'm sending commands for initiating video call, canceling video session, turning on/off camera,... Then on the other side I'm parsing these commands and executing functions.

function sendMutedMicCommand() { masterConnection.send(`${commands.MutedMic}`); }
function sendUnmutedMicCommand() { masterConnection.send(`${commands.UnmutedMic}`); }
function sendPromptVideoCallCommand() { masterConnection.send(`${commands.PromptVideoCall}`); }
function sendAcceptVideoCallCommand() { masterConnection.send(`${commands.AcceptVideoCall}`); }
function sendDeclineVideoCallCommand() { masterConnection.send(`${commands.DeclineVideoCall}`); }

Function which handles data:
function handleData(data) {
    let actionType = data;
    switch (actionType) {
        case commands.MutedMic: ShowMuteIconOnReceivingVideo(true); break;
        case commands.UnmutedMic: ShowMuteIconOnReceivingVideo(false); break;
        case commands.PromptVideoCall: showVideoCallModal(); break;
        case commands.AcceptVideoCall: startVideoConference(); break;
        case commands.DeclineVideoCall: showDeclinedCallAlert(); break;
        default: break;
    }
}


const commands = {
    MutedMic: "mutedMic",
    UnmutedMic: "unmutedMic",
    PromptVideoCall: "promptVideoCall",
    AcceptVideoCall: "acceptVideoCall",
    DeclineVideoCall: "declineVideoCall",
}

然后当我收到 mutedMic 命令时，我会显示带有交叉麦克风的图标.当我收到 AcceptVideoCall 命令时，我创建了另一个具有随机 ID 的对等端 videoCallPeer，然后将其发送到另一端.另一端然后创建另一个具有随机 ID 的对等点并使用接收到的 ID 发起视频会话.

And then when I receive mutedMic command, I show icon with crossed mic. When I receive AcceptVideoCall command I create another peer, videoCallPeer with random ID, which is then then sent to other side. Other side then created another peer with random ID and initiated video session with received ID.

这篇关于检测 MediaStreamTrack 是否为黑色/空白的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！