Http2&档案下载 [英] Http2 & File Download

问题描述

我们提供文件托管解决方案.我们的客户是最终用户，他们通过HTTP 1.1协议&下载文件.这些客户端基本上是软件系统或CDN，它们使用软件库下载我们的文件.没有人类用户访问我们的系统.我们还提供了使用HTTP/1.1范围标题等进行部分文件下载的选项.客户端系统还通过使用多个线程拆分成多个块来下载大文件.

We provide a file hosting solution. Our client are the end-users, who hit our servers though HTTP 1.1 protocol & download files. These client are basically software systems or CDNs, who download our files using software libraries. No human user accesses our system. We also provide option of partial file download using HTTP/1.1 range-header etc. Client system also download big file by splitting across chunks, using multiple threads.

我想检查一下，如果我们对服务器开放HTTP/2协议是否会有真正的好处?由于我们的客户端系统已经具有使用多个线程下载文件的功能，因此Http/2多路复用会带来真正的好处吗?

I want to check if there would be real benefit if we open up to HTTP/2 protocol to our servers? Since our client systems already have capability to use multiple threads to download our files, will Http/2 multiplexing will add any real benefit?

谢谢

推荐答案

HTTP/2文件的下载速度比HTTP/1.1慢，主要有两个原因:帧开销和

HTTP/2 file download is a bit slower than HTTP/1.1 for two main reasons: frame overhead and flow control.

在HTTP/1.1中，如果使用Content-Length分隔下载，则唯一下载的字节是内容字节. 但是，在HTTP/2中，每个DATA帧都为帧头携带9个额外的字节.在正常的最大帧大小为16384字节的情况下，这是一个很小的开销，但是仍然存在.

In HTTP/1.1, if you use Content-Length delimited downloads, the only bytes that are downloaded are the content bytes. In HTTP/2, however, each DATA frame carries 9 extra bytes for the frame header. At the normal max frame size of 16384 bytes that is a small overhead, yet it is present.

HTTP/2流控制是造成可能的速度下降的最大原因. 客户端必须确保扩大默认的会话和流流控制窗口，默认情况下均为65535字节.

The bigger contributor to possible slow downs is HTTP/2 flow control. Client must be sure to enlarge the default session and stream flow control windows, by default both at 65535 bytes.

HTTP/2的工作方式是服务器为每个HTTP/2会话(连接)和该会话中的每个流保留一个 send 窗口. 下载开始时，服务器有权为该流或该会话仅发送发送窗口允许的字节数，以先到者为准.然后，它必须等待客户端发送WINDOW_UPDATE帧，以补充流和会话流控制窗口，告诉服务器客户端已准备好接收更多数据.

The way HTTP/2 works is that the server keeps a send window for each HTTP/2 session (connection) and for each stream in that session. When the download begins, the server is entitled to send only a number of bytes allowed by the send window, for that stream or that session, whichever exhausts first. Then it has to wait for the client to send WINDOW_UPDATE frames, that replenish both the stream and session flow control windows, telling the server that the client is ready to receive more data.

对于小窗口(例如默认窗口)，此机制可能会由于客户端和服务器之间的网络延迟而导致下载性能下降，特别是如果天真地实施该机制. 在大多数情况下，服务器将处于等待状态，以等待客户端发送WINDOW_UPDATE，以便服务器可以发送更多数据.

For small windows such as the default ones, this mechanism may kill the download performance due to the network latency between client and server, especially if it is implemented naively. The server will be stalled most of the time waiting for the client to send a WINDOW_UPDATE so that the server can send more data.

多路复用扮演双重角色.虽然它允许并发启动许多文件的下载(可能比HTTP/1.1多得多的文件，但可能由于它只能打开较少数量的连接而受到限制)，但确实为每个流下载了数据有助于减少会话发送窗口.每个流可能仍具有未耗尽的发送窗口(因此它可以发送更多数据)，但是会话窗口已耗尽，因此服务器必须停止.流彼此竞争以消耗会话发送窗口.服务器实现也很重要，因为它必须正确地交错多个流中的帧.

Multiplexing plays a double role. While it allows to initiate the download of many files concurrently (possibly many more files than HTTP/1.1, that may be limited by the fact that it can only open a smaller number of connections), it is also true that data downloaded for each stream contributes to reduce the session send window. Each stream may still have a non exhausted send window (and therefore it could send more data), but the session window is exhausted and so the server must stall. The streams are competing with each other to consume the session send window. The server implementation is also important because it has to correctly interleave frames from multiple streams.

话虽如此，只要您对客户端和服务器都有相当高级的实现，并且有足够的调整旋钮来控制关键节点，HTTP/2仍然有可能与HTTP/1.1实现奇偶校验参数.

Having said that, it is still possible for HTTP/2 to achieve parity with HTTP/1.1, provided that you have a pretty advanced implementation of both the client and the server, and that you have enough tuning knobs to control the critical parameters.

理想情况下，在客户端上:

Ideally, on the client:

• 控制会话和流初始流控制窗口的能力
• 一个很好的实现，可以在服务器仍在下载时将WINDOW_UPDATE帧发送到服务器，以使服务器永不停顿；这可能需要自调整功能，具体取决于带宽延迟产品(类似于TCP有)

• ability to control the session and stream initial flow control windows
• a good implementation that sends WINDOW_UPDATE frames to the server while the server is still downloading, so that the server never stalls; this may require self-tuning features depending on the bandwidth-delay product (similarly to what TCP does)

理想情况下，在服务器上:

Ideally, on the server:

能够正确交错来自同一会话的多个流的帧(例如，避免下载第一个流的所有帧，然后下载第二个流的所有帧，等等，但是要先下载第一个流的一个帧，然后再下载一个)第二个视频流的帧，然后是第一个视频流的一帧，依此类推)

• ability to correctly interleave frames from multiple streams of the same session (e.g. avoid to download all frames of the first stream, then all the frames of the second stream, etc., but rather one frame of the first stream followed by one frame of the second stream, then again one frame of the first stream, etc.)

[免责声明，我是 Jetty ]的HTTP/2维护者

[Disclaimer, I am the HTTP/2 maintainer of Jetty]

Jetty 9.4.x支持上述所有功能，因为我们已经与社区和客户合作，以​​确保HTTP/2下载尽可能快.

Jetty 9.4.x supports all the features above, since we have worked with the community and customers to make sure that HTTP/2 downloads are as fast as possible.

我们在服务器上实现了适当的交错，Jetty的HttpClient和HTTP2Client分别提供了高级API和低级API，以处理HTTP和HTTP/2请求.在 BufferingFlowControlStrategy 并允许调整何时发送WINDOW_UPDATE帧(尽管尚未动态发送). 客户端还具有用于配置初始流控制窗口的选项. Jetty中的所有内容都是可插入的，因此您可以编写甚至更高级的流控制策略.

We implemented proper interleaving on the server, and Jetty's HttpClient and HTTP2Client provide respectively high level and low level APIs to deal with HTTP and HTTP/2 requests. The flow control is implemented in BufferingFlowControlStrategy and allows to tune when WINDOW_UPDATE frames are sent (although not yet dynamically). The clients also have options to configure the initial flow control windows. Everything in Jetty is pluggable, so you may write even more advanced flow control strategies.

即使您不使用Java或Jetty，也请确保在客户端和服务器上都剖析(或编写)正在使用的库，以便它们提供上述功能.

Even if you don't use Java or Jetty, make sure to dissect (or write) the libraries you're using on both the client and the server so that they provide the features mentioned above.

最后，您需要尝试进行测量；通过正确的HTTP/2实现和配置，多路复用效果应发挥作用，因此可以提高并行度并减少客户端和服务器上的资源利用率，因此您将比HTTP/1.1更具优势.

Finally, you need to try and measure; with a proper HTTP/2 implementation and configuration, the multiplexing effects should come into play, therefore increasing parallelism and reducing resource utilization on both client and server, so that you will have an advantage over HTTP/1.1.

这篇关于Http2&档案下载的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！