分块 WebSocket 传输 [英] Chunking WebSocket Transmission

问题描述

因为我在更常规的基础上使用 WebSocket 连接，所以我对引擎盖下的工作原理很感兴趣.所以我深入研究了无休止的规范文档一段时间，但到目前为止我真的找不到任何关于分块传输流本身的内容.

since I'm using WebSocket connections on more regular bases, I was interested in how things work under the hood. So I digged into the endless spec documents for a while, but so far I couldn't really find anything about chunking the transmission stream itself.

WebSocket 协议称其为数据帧(它描述的是纯数据流，因此也称为非控制帧).据我了解规范，没有定义的最大长度和 MTU(最大传输单位)值，这反过来意味着单个 WebSocket 数据帧可能包含，通过规范(！)，无限量的数据(如果我在这里说错了，请纠正我，我仍然是这方面的学生).

The WebSocket protocol calls it data frames (which describes the pure data stream, so its also called non-control frames). As far as I understood the spec, there is no defined max-length and no defined MTU (maximum transfer unit) value, that in turn means a single WebSocket data-frame may contain, by spec(!), an infinite amount of data (please correct me if I'm wrong here, I'm still a student on this).

阅读之后，我立即设置了我的小型 Node WebSocket 服务器.由于我有很强的Ajax 历史(也在流媒体和 Comet 上)，我最初的期望是，传输数据时必须有某种交互模式来读取数据".但我错了，不是吗?

After reading that, I instantly setup my little Node WebSocket server. Since I have a strong Ajax history (also on streaming and Comet), my expectations originaly were like, "there must be some kind of interactive mode for reading data while it is transfered". But I am wrong there, ain't I ?

我开始时很小，只有 4kb 的数据.

I started out small, with 4kb of data.

服务器

testSocket.emit( 'data', new Array( 4096 ).join( 'X' ) );

和预期的一样，它作为一个数据块到达客户端

and like expected this arrives on the client as one data-chunk

客户

wsInstance.onmessage = function( data ) {
    console.log( data.length ); // 4095
};

所以我增加了有效负载，实际上我再次期待，在某个时刻，客户端onmessage 处理程序将重复触发，有效地对传输进行分块.但令我震惊的是，它从未发生过(node-server，在 firefox、chrome 和 safari 客户端上测试-边).我最大的负载是80 MB

so I increased the payload and I actually was expecting again, that at some point, the client-side onmessage handler will fire repeatly, effectivley chunking the transmission. But to my shock, it never happened (node-server, tested on firefox, chrome and safari client-side). My biggest payload was 80 MB

testSocket.emit( 'data', new Array( 1024*1024*80 ).join( 'X' ) );

它仍然到达客户端的一个大数据块中.当然，即使您的连接非常好，这也需要一段时间.这里的问题是

and it still arrived in one big data-chunk on the client. Of course, this takes a while even if you have a pretty good connection. Questions here are

• 是否有可能将这些流分块，类似于 XHR readyState3 模式?
• 单个 ws 数据框是否有大小限制?
• websockets 不应该传输这么大的负载吗?(这会让我再次想知道为什么没有定义的最大尺寸)

我可能仍然从错误的角度看待 WebSockets，可能不存在发送大量数据的需求，您应该在发送之前自己对任何数据进行逻辑分块/拆分?

I might still look from the wrong perspective on WebSockets, probably the need for sending large data-amounts is just not there and you should chunk/split any data logically yourself before sending ?

推荐答案

首先，您需要区分浏览器中的 WebSocket 协议 和 WebSocket API.

First, you need to differentiate between the WebSocket protocol and the WebSocket API within browsers.

WebSocket 协议的帧大小限制为 2^63 个八位字节，但 WebSocket 消息可以由无限数量的帧组成.

The WebSocket protocol has a frame-size limit of 2^63 octets, but a WebSocket message can be composed of an unlimited number of frames.

浏览器中的 WebSocket API 不公开基于帧或流的 API，而仅公开基于消息的 API.传入消息的负载在提供给 JavaScript 之前总是完全缓冲(在浏览器的 WebSocket 实现中).

The WebSocket API within browsers does not expose a frame-based or streaming API, but only a message-based API. The payload of an incoming message is always completely buffered up (within the browser's WebSocket implementation) before providing it to JavaScript.

其他 WebSocket 实现的 API 可以提供对通过 WebSocket 协议传输的有效负载的基于帧或流的访问.例如，AutobahnPython 就可以.您可以在此处的示例中阅读更多内容 https://github.com/tavendo/AutobahnPython/tree/master/examples/twisted/websocket/streaming.

APIs of other WebSocket implementations may provide frame- or streaming-based access to payload transferred via the WebSocket protocol. For example, AutobahnPython does. You can read more in the examples here https://github.com/tavendo/AutobahnPython/tree/master/examples/twisted/websocket/streaming.

披露:我是 Autobahn 的原作者，为 Tavendo 工作.

Disclosure: I am original author of Autobahn and work for Tavendo.

更多注意事项:

只要浏览器JS WebSocket API中没有frame/streaming API，就只能接收/发送完整的WS消息.

As long as there is no frame/streaming API in browser JS WebSocket API, you can only receive/send complete WS messages.

单个(普通)WebSocket 连接不能交错多个消息的负载.也就是说，如果您使用大消息，这些消息会按顺序传递，并且当大消息仍在传输中时，您将无法在两者之间发送小消息.

A single (plain) WebSocket connection cannot interleave the payload of multiple messages. So i.e. if you use large messages, those are delivered in order, and you won't be able to send small messages in between while a big message is still on the fly.

有一个即将到来的 WebSocket 扩展(扩展是扩展协议的内置机制):WebSocket 多路复用.这允许在单个底层 TCP 连接上拥有多个(逻辑)WebSocket 连接，这具有多种优势.

There is an upcoming WebSocket extension (extensions are a builtin mechanism to extend the protocol): WebSocket multiplexing. This allows to have multiple (logical) WebSocket connections over a single underlying TCP connection, which has multiple advantages.

另请注意:您可以从单个 JS/HTML 页面今天打开到单个目标服务器的多个 WS 连接(通过不同的底层 TCP).

Note also: you can open multiple WS connections (over different underlying TCPs) to a single target server from a single JS / HTML page today.

另请注意:您可以在应用程序层对自己进行分块":在较小的 WS 消息中发送您的内容，然后自己重​​新组装.

Note also: you can do "chunking" yourself in application layer: send your stuff in smaller WS messages a reassemble yourself.

我同意，在理想情况下，您应该在浏览器中拥有消息/帧/流 API 以及 WebSocket 多路复用.这将提供所有的力量和便利.

I agree, in an ideal world, you'd have message/frame/streaming API in browser plus WebSocket multiplexing. That would give all the power and convenience.

这篇关于分块 WebSocket 传输的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！