远程VideoStream无法与WebRTC一起使用 [英] Remote VideoStream not working with WebRTC
问题描述
我写了一篇详细的教程，解释了如何构建一个包括信令服务器的简单Videochat应用程序:
I wrote a detailed tutorial explaining how to build an simple Videochat-application including a signaling server:
请告诉我您是否有帮助&可以理解的.谢谢！
Please tell me if you find it helpful & understandable. Thanks!
我正在尝试通过WebRTC和Websocket(nodejs服务器)使Streams正常工作.据我所知，通过SDP进行的握手有效，并建立了Peerconnection. 问题是-远程视频无法播放. src-Attribute获取Blob并设置了自动播放功能，但不会播放. 也许我对ICE候选者做错了(它们用于媒体流，对吗?). 有什么方法可以检查PeerConnection的设置是否正确?
i am trying to get Streams to work via WebRTC and Websocket (nodejs-server). As far as i can see the handshake via SDP works and the Peerconnection is established. The problem is - the Remote-Video is not playing. The src-Attribute gets the Blob and autoplay is set, but it just won`t play. Maybe i am doing something wrong with the ICE-candidates (they are used for media-streaming, right?). Is there any way to check if the PeerConnection is set up correctly?
也许我应该解释代码的工作原理
Maybe i should explain how the code works
-
在网站加载时，建立了与websocket服务器的连接，使用Google STUN服务器创建了PeerConnection，并收集了视频和音频流&添加到PeerConnection
At load of website a connection to the websocket-server is established, an PeerConnection using googles STUN-server is created and Video and Audio-Streams are collected & added to the PeerConnection
当一个用户单击创建要约"按钮时，包含其会话描述(SDP)的消息将发送到服务器(客户端func sendOffer())，该消息会广播给另一用户
When one user clicks on "create offer"-button a message containing its Session-Description (SDP) is send to the server (client func sendOffer()), which broadcasts it to the other user
其他用户收到消息并保存他收到的SDP
The other user gets the message and saves the SDP he received
如果用户单击接受报价"，则将SDP添加到RemoteDescription(func createAnswer())中，然后将应答消息(包含应答用户的SDP)发送给报价用户
If the user clicks "accept offer", the SDP is added to the RemoteDescription (func createAnswer()) which then sends an answer-message (containing the SDP of the answering-user) to the offering-user
在提供用户侧执行func offerAccepted()，它将另一个用户的SDP添加到他的RemoteDesription中.
At the offering-user`s side the func offerAccepted() is executed, which adds the SDP of the other user to his RemoteDesription.
我不确定在什么时候准确地称为icecandidate-handler，但是我认为它们应该起作用，因为我两面都有日志.
I am not sure at what point exactly the icecandidate-handlers are called, but i think they should work because i get both logs on both sides.
这是我的代码(这只是用于测试，因此，即使有一个称为广播的功能，也意味着一次只能有2个用户访问同一网站)
Here`s my Code (this is just for testing, so even if there is a function called broadcast it means that only 2 users can be on the same website at a time):
index.html的标记:
<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8">
<style>
#acceptOffer {
display: none;
}
</style>
</head>
<body>
<h2>Chat</h2>
<div>
<textarea class="output" name="" id="" cols="30" rows="10"></textarea>
</div>
<button id="createOffer">create Offer</button>
<button id="acceptOffer">accept Offer</button>
<h2>My Stream</h2>
<video id="myStream" autoplay src=""></video>
<h2>Remote Stream</h2>
<video id="remoteStream" autoplay src=""></video>
<script src="http://ajax.googleapis.com/ajax/libs/jquery/1.9.1/jquery.min.js"></script>
<script src="websocketClient.js"></script>
</body>
</html>
这是服务器代码:
"use strict";
var webSocketsServerPort = 61122;
var webSocketServer = require('websocket').server,
http = require('http'),
clients = [];
var server = http.createServer(function(request, response) {
// Not important for us. We're writing WebSocket server, not HTTP server
});
server.listen(webSocketsServerPort, function() {
console.log((new Date()) + " Server is listening on port " + webSocketsServerPort);
});
var wsServer = new webSocketServer({
httpServer: server
});
wsServer.on('request', function(request) {
console.log((new Date()) + ' Connection from origin ' + request.origin + '.');
var connection = request.accept(null, request.origin),
index = clients.push(connection) - 1,
userName=false;
console.log((new Date()) + ' Connection accepted from '+connection.remoteAddress);
// user sent some message
connection.on('message', function(message) {
var json = JSON.parse(message.utf8Data);
console.log(json.type);
switch (json.type) {
case 'broadcast':
broadcast(json);
break;
case 'emit':
emit({type:'offer', data:json.data.data});
break;
case 'client':
respondToClient(json, clients[index]);
break;
default:
respondToClient({type:'error', data:'Sorry, i dont understand that.'}, clients[index]);
break;
}
});
connection.on('close', function(connection) {
clients.splice(index,1);
console.log((new Date()) + " Peer " + connection.remoteAddress + " disconnected.");
broadcast({type:'text', data: userName+' has left the channel.'});
});
var respondToClient = function(data, client){
client.sendUTF(JSON.stringify( data ));
};
var broadcast = function(data){
for(var i = 0; i < clients.length; i++ ) {
if(i != index ) {
clients[i].sendUTF(JSON.stringify( data ));
}
}
};
var emit = function(){
// TBD
};
});
这里是客户端代码:
$(function () {
"use strict";
/**
* Websocket Stuff
**/
window.WebSocket = window.WebSocket || window.MozWebSocket;
// open connection
var connection = new WebSocket('ws://url-to-node-server:61122'),
myName = false,
mySDP = false,
otherSDP = false;
connection.onopen = function () {
console.log("connection to WebSocketServer successfull");
};
connection.onerror = function (error) {
console.log("WebSocket connection error");
};
connection.onmessage = function (message) {
try {
var json = JSON.parse(message.data),
output = document.getElementsByClassName('output')[0];
switch(json.callback) {
case 'offer':
otherSDP = json.data;
document.getElementById('acceptOffer').style.display = 'block';
break;
case 'setIceCandidate':
console.log('ICE CANDITATE ADDED');
peerConnection.addIceCandidate(json.data);
break;
case 'text':
var text = output.value;
output.value = json.data+'\n'+output.value;
break;
case 'answer':
otherSDP = json.data;
offerAccepted();
break;
}
} catch (e) {
console.log('This doesn\'t look like a valid JSON or something else went wrong.');
return;
}
};
/**
* P2P Stuff
**/
navigator.getMedia = ( navigator.getUserMedia ||
navigator.webkitGetUserMedia ||
navigator.mozGetUserMedia ||
navigator.msGetUserMedia);
// create Connection
var peerConnection = new webkitRTCPeerConnection(
{ "iceServers": [{ "url": "stun:stun.l.google.com:19302" }] }
);
var remoteVideo = document.getElementById('remoteStream'),
myVideo = document.getElementById('myStream'),
// get local video-Stream and add to Peerconnection
stream = navigator.webkitGetUserMedia({ audio: false, video: true }, function (stream) {
myVideo.src = webkitURL.createObjectURL(stream);
console.log(stream);
peerConnection.addStream(stream);
});
// executes if other side adds stream
peerConnection.onaddstream = function(e){
console.log("stream added");
if (!e)
{
return;
}
remoteVideo.setAttribute("src",URL.createObjectURL(e.stream));
console.log(e.stream);
};
// executes if my icecandidate is received, then send it to other side
peerConnection.onicecandidate = function(candidate){
console.log('ICE CANDITATE RECEIVED');
var json = JSON.stringify( { type: 'broadcast', callback:'setIceCandidate', data:candidate});
connection.send(json);
};
// send offer via Websocket
var sendOffer = function(){
peerConnection.createOffer(function (sessionDescription) {
peerConnection.setLocalDescription(sessionDescription);
// POST-Offer-SDP-For-Other-Peer(sessionDescription.sdp, sessionDescription.type);
var json = JSON.stringify( { type: 'broadcast', callback:'offer',data:{sdp:sessionDescription.sdp,type:'offer'}});
connection.send(json);
}, null, { 'mandatory': { 'OfferToReceiveAudio': true, 'OfferToReceiveVideo': true } });
};
// executes if offer is received and has been accepted
var createAnswer = function(){
peerConnection.setRemoteDescription(new RTCSessionDescription(otherSDP));
peerConnection.createAnswer(function (sessionDescription) {
peerConnection.setLocalDescription(sessionDescription);
// POST-answer-SDP-back-to-Offerer(sessionDescription.sdp, sessionDescription.type);
var json = JSON.stringify( { type: 'broadcast', callback:'answer',data:{sdp:sessionDescription.sdp,type:'answer'}});
connection.send(json);
}, null, { 'mandatory': { 'OfferToReceiveAudio': true, 'OfferToReceiveVideo': true } });
};
// executes if other side accepted my offer
var offerAccepted = function(){
peerConnection.setRemoteDescription(new RTCSessionDescription(otherSDP));
console.log('it should work now');
};
$('#acceptOffer').on('click',function(){
createAnswer();
});
$('#createOffer').on('click',function(){
sendOffer();
});
});
我还读到，在发送任何要约之前，必须先收集本地媒体流.创建PeerConnection时是否必须添加它? IE.像这样的东西:
I also read that the local-media-stream has to be collected before any offer is send. Does it mean i have to add it when the PeerConnection is created? I.e. something like this:
// create Connection
var peerConnection = new webkitRTCPeerConnection(
{
"iceServers": [{ "url": "stun:stun.l.google.com:19302" }],
"mediaStream": stream // attach media stream here?
}
);
在此先感谢您的帮助！
我现在再远一点.似乎由于指定了无效或非法的字符串."而无法添加远程制冰候选对象(客户端代码中的切换用例setIceCandidate). json.data.candidate-object看起来像这样:
i am a bit further now. it seems that adding the remote ice-candidates (switch-case setIceCandidate in client-code) is not working because of "An invalid or illegal string was specified. ". the json.data.candidate-object looks like this:
candidate: "a=candidate:1663431597 2 udp 1845501695 141.84.69.86 57538 typ srflx raddr 10.150.16.92 rport 57538 generation 0
↵"
sdpMLineIndex: 1
sdpMid: "video"
我试图创建一个像这样的新候选人
i tried creating an new candidate like this
var remoteCandidate = new RTCIceCandidate(json.data.candidate);
peerConnection.addIceCandidate(remoteCandidate);
但是我仍然遇到语法错误
but i still got an syntax error
推荐答案
最近我在处理同一件事上遇到了麻烦，而我从这里的其他人那里得到的最佳建议是创建程序的一个版本，手动将SDP和ICE信息从一个对等"(即浏览器选项卡)复制并粘贴到另一个，反之亦然.
I was having trouble with essentially the same thing recently, and the best advice I got from someone else on here was to create a version of my program in which I manually copied and pasted the SDP and ICE info from one "peer" (i.e., browser tab) to another and vice versa.
这样做，我意识到了几件事:
By doing this, I realized several things:
-
在尝试创建任何要约/答案之前，必须先调用对等连接对象 的addStream方法.
You must call the addStream method of the peer connection object before you attempt to create any offers/answers.
调用createOffer或createAnswer方法后，将立即生成该客户端的ICE候选对象.但是，一旦将ICE信息发送给其他对等方，就无法真正设置ICE信息，直到设置了远程描述之后(使用收到的要约/答案).
Upon calling the createOffer or createAnswer method, the ICE candidates for that client are instantly generated. However, once you've sent the ICE info to the other peer, you cannot actually set the ICE info until after a remote description is set (by using the received offer/answer).
请确保您正确编码了有关将在网络上发送的所有信息.在JS中，这意味着您应该对将要在网络上发送的所有数据使用encodeURIComponent函数.我有一个问题，其中SDP和ICE信息有时会被正确设置，而有时却未正确设置.这与以下事实有关:我没有对数据进行URI编码，这导致数据中的任何加号都变成空格，从而使所有内容弄乱了.
Make sure that you are properly encoding all info about to be sent on the wire. In JS, this means that you should use the encodeURIComponent function on all data about to be sent on the wire. I had an issue in which SDP and ICE info would sometimes be set properly and sometimes not. It had to do with the fact that I was not URI-encoding the data, which led to any plus signs in the data being turned into spaces, which messed everything up.
无论如何，就像我说的那样，我建议创建一个程序版本，在该版本中，您有一堆文本区域可用于将所有数据分散到屏幕上，然后还有其他文本区域可将复制的数据粘贴到其中.将其设置为其他同级.
这样做确实澄清了整个WebRTC流程，说实话，在我所见过的任何文档/教程中都没有对它进行很好的解释.
Anyway, like I said, I recommend creating a version of your program in which you have a bunch of text areas for spitting out all the data to the screen, and then have other text areas that you can paste copied data into for setting it for the other peer.
Doing this really clarified the whole WebRTC process, which honestly, is not well explained in any documents/tutorials that I've seen yet.
祝你好运，请告诉我是否能再提供帮助.
Good luck, and let me know if I can help anymore.
这篇关于远程VideoStream无法与WebRTC一起使用的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
