分块WebSocket传输 [英] Chunking WebSocket Transmission

问题描述

因为我在更常规的基础上使用WebSocket连接，所以我对如何在幕后工作感兴趣。所以我在一段时间内深入研究了无休止的规范文档，但到目前为止，我无法找到关于分块传输流本身的任何内容。

since I'm using WebSocket connections on more regular bases, I was interested in how things work under the hood. So I digged into the endless spec documents for a while, but so far I couldn't really find anything about chunking the transmission stream itself.

WebSocket协议将其称为数据帧（描述纯数据流，因此它也称为非控制帧）。据我了解规范，没有定义的最大长度和没有定义的MTU（最大传输单位）值，这反过来意味着单个WebSocket数据框可能包含spec（！），无限量的数据（如果我错了，请纠正我，我仍然是这个学生。）

The WebSocket protocol calls it data frames (which describes the pure data stream, so its also called non-control frames). As far as I understood the spec, there is no defined max-length and no defined MTU (maximum transfer unit) value, that in turn means a single WebSocket data-frame may contain, by spec(!), an infinite amount of data (please correct me if I'm wrong here, I'm still a student on this).

阅读完毕后，我立即设置了我的小 Node WebSocket服务器。由于我有一个强大的 Ajax 历史（也在流媒体和Comet上），我的期望原来就像是，在传输数据时必须有某种交互模式来读取数据。但我错了，不是吗？

After reading that, I instantly setup my little Node WebSocket server. Since I have a strong Ajax history (also on streaming and Comet), my expectations originaly were like, "there must be some kind of interactive mode for reading data while it is transfered". But I am wrong there, ain't I ?

我开始时很小，有 4kb 的数据。

I started out small, with 4kb of data.

服务器

testSocket.emit( 'data', new Array( 4096 ).join( 'X' ) );

和预期的一样，它作为一个数据块到达客户端

and like expected this arrives on the client as one data-chunk

客户

wsInstance.onmessage = function( data ) {
    console.log( data.length ); // 4095
};

所以我增加了有效载荷，我实际上再次期待，在某些时候，客户端 onmessage 处理程序将重复触发，effectivley对传输进行分块。但令我震惊的是，它从未发生过（ node-server ，在 firefox ， chrome 和 safari 客户端上测试过 - 侧）。我最大的有效载荷是 80 MB

so I increased the payload and I actually was expecting again, that at some point, the client-side onmessage handler will fire repeatly, effectivley chunking the transmission. But to my shock, it never happened (node-server, tested on firefox, chrome and safari client-side). My biggest payload was 80 MB

testSocket.emit( 'data', new Array( 1024*1024*80 ).join( 'X' ) );

它仍然存在于客户端的一个大数据块中。当然，即使你有一个非常好的连接，这需要一段时间。这里的问题是

and it still arrived in one big data-chunk on the client. Of course, this takes a while even if you have a pretty good connection. Questions here are

• 是否有可能将这些流分块，类似于XHR readyState3模式？


• 单个ws数据框有任何大小限制吗？


• 是不应该传输如此大的有效载荷的websockets吗？ （这会让我再次想知道为什么没有定义的最大尺寸）

• is there any possiblity to chunk those streams, similar to the XHR readyState3 mode ?
• is there any size limit for a single ws data-frame ?
• are websockets not supposed to transfer such large payloads? (which would make me wonder again why there isn't a defined max-size)

我可能仍然会从对WebSockets的错误观点，可能是发送大量数据的需要不存在，你应该在发送之前自己逻辑地分块/拆分任何数据吗？

I might still look from the wrong perspective on WebSockets, probably the need for sending large data-amounts is just not there and you should chunk/split any data logically yourself before sending ?

推荐答案

首先，您需要在 browsers 中区分WebSocket 协议和WebSocket API 。

First, you need to differentiate between the WebSocket protocol and the WebSocket API within browsers.

WebSocket协议的帧大小限制为2 ^ 63个八位字节，但WebSocket消息可以由无限数量的帧组成。

The WebSocket protocol has a frame-size limit of 2^63 octets, but a WebSocket message can be composed of an unlimited number of frames.

浏览器中的WebSocket API不公开基于帧的API或流API，而只公开基于消息的API。在将消息提供给JavaScript之前，传入消息的有效负载总是被完全缓冲（在浏览器的WebSocket实现中）。

The WebSocket API within browsers does not expose a frame-based or streaming API, but only a message-based API. The payload of an incoming message is always completely buffered up (within the browser's WebSocket implementation) before providing it to JavaScript.

其他WebSocket实现的API可能提供帧或流基于WebSocket协议传输的有效载荷访问。例如， AutobahnPython 可以。您可以在此处的示例中阅读更多内容 https://github.com/tavendo/AutobahnPython/ tree / master / examples / twisted / websocket / streaming 。

APIs of other WebSocket implementations may provide frame- or streaming-based access to payload transferred via the WebSocket protocol. For example, AutobahnPython does. You can read more in the examples here https://github.com/tavendo/AutobahnPython/tree/master/examples/twisted/websocket/streaming.

披露：我是Autobahn的原作者，为Tavendo工作。

Disclosure: I am original author of Autobahn and work for Tavendo.

更多注意事项：

只要浏览器JS WebSocket API中没有帧/流API，您就只能接收/发送完整WS消息。

As long as there is no frame/streaming API in browser JS WebSocket API, you can only receive/send complete WS messages.

单个（普通）WebSocket连接不能交错多个消息的有效负载。因此，即如果您使用大型邮件，这些邮件将按顺序发送，并且您仍然无法在大邮件仍在运行时发送小邮件。

A single (plain) WebSocket connection cannot interleave the payload of multiple messages. So i.e. if you use large messages, those are delivered in order, and you won't be able to send small messages in between while a big message is still on the fly.

即将推出的WebSocket扩展（扩展是扩展协议的内置机制）：WebSocket多路复用。这允许在单个底层TCP连接上具有多个（逻辑）WebSocket连接，这具有多个优点。

There is an upcoming WebSocket extension (extensions are a builtin mechanism to extend the protocol): WebSocket multiplexing. This allows to have multiple (logical) WebSocket connections over a single underlying TCP connection, which has multiple advantages.

另请注意：您可以打开多个WS连接（通过不同的底层连接） TCPs）从单个JS / HTML页面今天到单个目标服务器。

Note also: you can open multiple WS connections (over different underlying TCPs) to a single target server from a single JS / HTML page today.

另请注意：您可以在应用程序中自行分块 layer：在较小的WS消息中发送你的东西，然后重新组装。

Note also: you can do "chunking" yourself in application layer: send your stuff in smaller WS messages a reassemble yourself.

我同意，在理想的世界中，你在浏览器和WebSocket中都有消息/帧/流API复用。这将提供所有的力量和便利。

I agree, in an ideal world, you'd have message/frame/streaming API in browser plus WebSocket multiplexing. That would give all the power and convenience.

这篇关于分块WebSocket传输的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！