HTTP / 2和档案下载 [英] HTTP/2 & File Download

问题描述

我们提供文件托管解决方案。我们的客户是最终用户，他们通过HTTP 1.1协议&下载文件。这些客户端基本上是软件系统或CDN，它们使用软件库下载我们的文件。没有人类用户访问我们的系统。我们还提供了使用HTTP / 1.1范围标题等进行部分文件下载的选项。客户端系统还通过使用多个线程拆分成多个块来下载大文件。

We provide a file hosting solution. Our client are the end-users, who hit our servers though HTTP 1.1 protocol & download files. These client are basically software systems or CDNs, who download our files using software libraries. No human user accesses our system. We also provide option of partial file download using HTTP/1.1 range-header etc. Client system also download big file by splitting across chunks, using multiple threads.

我要检查是否存在如果我们对服务器开放HTTP / 2.0协议，这将是真正的好处吗？由于我们的客户端系统已经具有使用多个线程下载文件的功能，HTTP / 2.0多路复用是否会带来真正的好处？

I want to check if there would be real benefit if we open up to HTTP/2.0 protocol to our servers? Since our client systems already have capability to use multiple threads to download our files, will HTTP/2.0 multiplexing will add any real benefit?

谢谢

推荐答案

HTTP / 2文件下载比HTTP / 1.1慢一点，这有两个主要原因：帧开销和流控制。

HTTP/2 file download is a bit slower than HTTP/1.1 for two main reasons: frame overhead and flow control.

在HTTP / 1.1中，如果使用内容长度分隔下载，唯一下载的字节是内容字节。
但是，在HTTP / 2中，每个 DATA 帧都为帧头携带9个额外的字节。在正常的最大帧大小为16384字节的情况下，这是一个很小的开销，但是仍然存在。

In HTTP/1.1, if you use Content-Length delimited downloads, the only bytes that are downloaded are the content bytes. In HTTP/2, however, each DATA frame carries 9 extra bytes for the frame header. At the normal max frame size of 16384 bytes that is a small overhead, yet it is present.

HTTP / 2流控制是造成速度下降的主要原因。
客户端必须确保扩大默认会话和流控制窗口，默认情况下均为65535字节。

The bigger contributor to possible slow downs is HTTP/2 flow control. Client must be sure to enlarge the default session and stream flow control windows, by default both at 65535 bytes.

HTTP / 2的工作方式是服务器为每个HTTP / 2会话（连接）和该会话中的每个流保留一个 send 窗口。
下载开始时，服务器有权为该流或该会话仅发送发送窗口允许的字节数，以先耗尽者为准。然后，它必须等待客户端发送 WINDOW_UPDATE 帧，这些帧会补充流和会话流控制窗口，告诉服务器客户端已准备好接收更多数据。

The way HTTP/2 works is that the server keeps a send window for each HTTP/2 session (connection) and for each stream in that session. When the download begins, the server is entitled to send only a number of bytes allowed by the send window, for that stream or that session, whichever exhausts first. Then it has to wait for the client to send WINDOW_UPDATE frames, that replenish both the stream and session flow control windows, telling the server that the client is ready to receive more data.

对于小窗口（例如默认窗口），此机制可能会由于客户端和服务器之间的网络延迟而导致下载性能下降，尤其是如果天真地实现。
服务器大部分时间都处于停滞状态，等待客户端发送 WINDOW_UPDATE ，以便服务器可以发送更多数据。

For small windows such as the default ones, this mechanism may kill the download performance due to the network latency between client and server, especially if it is implemented naively. The server will be stalled most of the time waiting for the client to send a WINDOW_UPDATE so that the server can send more data.

多路复用扮演双重角色。虽然它允许并发启动许多文件的下载（可能比HTTP / 1.1多得多的文件，但实际上它只能打开较少的连接数，因此可能受到限制），但确实为每个流下载了数据有助于减少会话发送窗口。每个流可能仍具有未耗尽的发送窗口（因此它可以发送更多数据），但是会话窗口已耗尽，因此服务器必须停止。流彼此竞争以消耗会话发送窗口。服务器实现也很重要，因为它必须正确地交错多个流中的帧。

Multiplexing plays a double role. While it allows to initiate the download of many files concurrently (possibly many more files than HTTP/1.1, that may be limited by the fact that it can only open a smaller number of connections), it is also true that data downloaded for each stream contributes to reduce the session send window. Each stream may still have a non exhausted send window (and therefore it could send more data), but the session window is exhausted and so the server must stall. The streams are competing with each other to consume the session send window. The server implementation is also important because it has to correctly interleave frames from multiple streams.

尽管如此，HTTP / 2仍可能与HTTP /实现奇偶校验1.1，前提是您对客户端和服务器均具有相当高级的实现，并且具有足够的调整旋钮来控制关键参数。

Having said that, it is still possible for HTTP/2 to achieve parity with HTTP/1.1, provided that you have a pretty advanced implementation of both the client and the server, and that you have enough tuning knobs to control the critical parameters.

理想情况下，客户：

• 控制会话和流初始流控制窗口的能力


• 一个很好的实现在服务器仍在下载时将 WINDOW_UPDATE 帧发送到服务器，以使服务器永不停止；这可能需要自调整功能，具体取决于带宽延迟产品（类似于TCP确实）

• ability to control the session and stream initial flow control windows
• a good implementation that sends WINDOW_UPDATE frames to the server while the server is still downloading, so that the server never stalls; this may require self-tuning features depending on the bandwidth-delay product (similarly to what TCP does)

理想情况下，在服务器上：

Ideally, on the server:

能够正确交错来自同一会话的多个流的帧（例如，避免下载第一个流的所有帧，然后下载第二个流的所有帧，等等，但是要先下载一个流的一个帧，然后再下载一个）第二个流的帧，然后是第一个流的一帧，依此类推。）

[免责声明，我是HTTP / 2 Jetty ]

[Disclaimer, I am the HTTP/2 maintainer of Jetty]

Jetty 9.4的维护者。 x支持上述所有功能，因为我们已经与社区和客户合作，以​​确保HTTP / 2下载尽可能快。

Jetty 9.4.x supports all the features above, since we have worked with the community and customers to make sure that HTTP/2 downloads are as fast as possible.

我们在服务器和Jetty的 Ht tpClient 和 HTTP2Client 分别提供了高级API和低级API来处理HTTP和HTTP / 2请求。流控制在 BufferingFlowControlStrategy 并允许在发送 WINDOW_UPDATE 帧时进行调整（尽管尚未动态）。
客户端还可以选择配置初始流控制窗口。
Jetty中的所有内容都是可插入的，因此您甚至可以编写更高级的流控制策略。

We implemented proper interleaving on the server, and Jetty's HttpClient and HTTP2Client provide respectively high level and low level APIs to deal with HTTP and HTTP/2 requests. The flow control is implemented in BufferingFlowControlStrategy and allows to tune when WINDOW_UPDATE frames are sent (although not yet dynamically). The clients also have options to configure the initial flow control windows. Everything in Jetty is pluggable, so you may write even more advanced flow control strategies.

即使您不使用Java或Jetty，也请确保剖析（或编写）您在客户端和服务器上使用的库，以便它们提供上述功能。

Even if you don't use Java or Jetty, make sure to dissect (or write) the libraries you're using on both the client and the server so that they provide the features mentioned above.

最后，您需要尝试并进行测量;通过正确的HTTP / 2实现和配置，多路复用效果应发挥作用，因此可以提高并行度并减少客户端和服务器上的资源利用率，因此您将比HTTP / 1.1更具优势。

Finally, you need to try and measure; with a proper HTTP/2 implementation and configuration, the multiplexing effects should come into play, therefore increasing parallelism and reducing resource utilization on both client and server, so that you will have an advantage over HTTP/1.1.

这篇关于HTTP / 2和档案下载的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！