Apache Http 客户端比浏览器下载慢 [英] Apache Http Client slower than browser download
问题描述
我正在从 github 下载一个大小为 300M 的大型存储库.从浏览器下载需要 10-15 秒.在同一台机器上,当我使用以下代码下载时需要 110-120 秒.我想知道我是否做错了.请建议我使用 apache http 客户端获得相同的速度(10-15 秒).或者有什么比 http 客户端更好的吗?
I am downloading a large repository from github of size 300M. It takes 10-15sec when I download from my browser. On the same machine, it takes 110-120sec when I use below code to download. I am wondering if I am doing wrong. Please suggest me to get the same speed(10-15sec) using apache http client. Or is there anything better than http client ?
Apache httpclient = 4.5
Apache httpclient = 4.5
Java - 8
我使用的代码:
import java.io.File;
import java.io.IOException;
import java.io.InputStream;
import java.net.URL;
import org.apache.commons.io.FileUtils;
import org.apache.commons.io.IOUtils;
import org.apache.http.HttpResponse;
import org.apache.http.client.ClientProtocolException;
import org.apache.http.client.ResponseHandler;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.impl.client.CloseableHttpClient;
import org.apache.http.impl.client.HttpClientBuilder;
import org.apache.http.impl.client.LaxRedirectStrategy;
import org.apache.http.impl.conn.PoolingHttpClientConnectionManager;
public class Downloader {
public File download(URL url, File dstFile) {
PoolingHttpClientConnectionManager manager = new PoolingHttpClientConnectionManager();
manager.setDefaultMaxPerRoute(20);
manager.setMaxTotal(200);
CloseableHttpClient httpclient = HttpClientBuilder.create()
.setConnectionManager(manager)
.build();
// Second option: it also takes same time.
// .setRedirectStrategy(new LaxRedirectStrategy())
// .setMaxConnTotal(2 * 50)
// .setMaxConnPerRoute(50)
// .build();
// CloseableHttpClient httpclient = HttpClients.custom()
// .setRedirectStrategy(new LaxRedirectStrategy()) // adds HTTP REDIRECT support to GET and POST methods
// .build();
try {
HttpGet get = new HttpGet(url.toURI()); // we're using GET but it could be via POST as well
File downloaded = httpclient.execute(get, new FileDownloadResponseHandler(dstFile));
return downloaded;
} catch (Exception e) {
throw new IllegalStateException(e);
} finally {
IOUtils.closeQuietly(httpclient);
}
}
static class FileDownloadResponseHandler implements ResponseHandler<File> {
private final File target;
public FileDownloadResponseHandler(File target) {
this.target = target;
}
@Override
public File handleResponse(HttpResponse response) throws ClientProtocolException, IOException {
InputStream source = response.getEntity().getContent();
FileUtils.copyInputStreamToFile(source, this.target);
return this.target;
}
}
}
推荐答案
我遇到了同样的问题;修复是从这里实施建议:https://stackoverflow.com/a/35458078/9204142,即使用 disableContentCompression()
或 contentCompressionEnabled=false
在 RequestConfig
级别.修复后速度与 curl 匹配.
I had the same problem; the fix was implementing the suggestion from here: https://stackoverflow.com/a/35458078/9204142, i.e. use disableContentCompression()
or contentCompressionEnabled=false
at the RequestConfig
level.
After the fix the speed matched the one of curl.
因为我使用了 Apache Camel,所以修复意味着在 Endpoint 的 URI 中添加 ?httpClient.contentCompressionEnabled=false
.
As I've used Apache Camel the fix implied adding ?httpClient.contentCompressionEnabled=false
in the URI of the Endpoint.
这篇关于Apache Http 客户端比浏览器下载慢的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!