okhttp 3:如何使用Java/Android手动解压缩gzip/deflate响应 [英] okhttp 3: how to decompress gzip/deflate response manually using Java/Android
问题描述
我知道okhttp3
库默认情况下会添加标头Accept-Encoding: gzip
并自动为我们解码响应.
I know that okhttp3
library by default it adds the header Accept-Encoding: gzip
and decodes the response automatically for us.
我正在处理的主机仅接受标头之类的问题:Accept-Encoding: gzip, deflate
如果不添加deflate
部分,它将失败.现在,当我手动将该标头添加到okhttp客户端时,该库不再对我进行解压缩.
The problem I'm dealing with a host that only accepts a header like: Accept-Encoding: gzip, deflate
if I don't add the deflate
part it fails. Now when I manually add that header to okhttp client, the library doesn't do the decompression anymore for me.
我已经尝试了多种解决方案来获取响应并尝试手动解压缩该响应,但是我总是最终遇到异常,例如java.util.zip.ZipException: Not in GZIP format
,这是我到目前为止已经尝试过的方法:
I've tried multiple solutions to take the response and try to manually decompress that but I've always ended up with an exception i.e. java.util.zip.ZipException: Not in GZIP format
, here's what I've tried so far:
//decompresser
public static String decompressGZIP(InputStream inputStream) throws IOException
{
InputStream bodyStream = new GZIPInputStream(inputStream);
ByteArrayOutputStream outStream = new ByteArrayOutputStream();
byte[] buffer = new byte[4096];
int length;
while ((length = bodyStream.read(buffer)) > 0)
{
outStream.write(buffer, 0, length);
}
return new String(outStream.toByteArray());
}
//run scraper
scrape(api, new Callback()
{
// Something went wrong
@Override
public void onFailure(@NonNull Call call, @NonNull IOException e)
{
}
@Override
public void onResponse(@NonNull Call call, @NonNull Response response) throws IOException
{
if (response.isSuccessful())
{
try
{
InputStream responseBodyBytes = responseBody.byteStream();
returnedObject = GZIPCompression.decompress(responseBodyBytes);
if (returnedObject != null)
{
String htmlResponse = returnedObject.toString();
}
}
catch (ProtocolException e){}
if(response != null) response.close();
}
}
});
private Call scrape(Map<?, ?> api, Callback callback)
{
MediaType JSON = MediaType.parse("application/json; charset=utf-8");
String method = (String) api.get("method");
String url = (String) api.get("url");
Request.Builder requestBuilder = new Request.Builder().url(url);
RequestBody requestBody;
requestBuilder.header("User-Agent", "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:61.0) Gecko/20100101 Firefox/61.0");
requestBuilder.header("Accept", "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8");
requestBuilder.header("Accept-Language", "en-US,en;q=0.5");
requestBuilder.header("Accept-Encoding", "gzip, deflate");
requestBuilder.header("Connection", "keep-alive");
requestBuilder.header("Upgrade-Insecure-Requests", "1");
requestBuilder.header("Cache-Control", "max-age=0");
Request request = requestBuilder.build();
Call call = client.newCall(request);
call.enqueue(callback);
return call;
}
仅需注意,响应头将始终返回Content-Encoding: gzip
和Transfer-Encoding: chunked
Just a note, the response headers will always return Content-Encoding: gzip
and Transfer-Encoding: chunked
还有一件事,我也在本主题中尝试了该解决方案,但对于D/OkHttp: java.io.IOException: ID1ID2: actual 0x00003c68 != expected 0x00001f8b
仍然失败.
One more thing, I've also tried the solution in this topic and it still fails with D/OkHttp: java.io.IOException: ID1ID2: actual 0x00003c68 != expected 0x00001f8b
.
任何帮助将不胜感激.
推荐答案
经过6个小时的挖掘,我找到了正确的解决方案,并且比往常想像的要容易,所以我基本上是在尝试解压缩未压缩的页面那就是失败的原因.现在,一旦我进入第二页(压缩的页面),我就会得到一个压缩的响应,上面的代码应该在该响应上进行处理.另外,如果有人想要解决方案,我就像此答案中的拦截器一样使用了经过改进的拦截器,因此您无需使用一个自定义函数来处理减压.
After 6 hours of digging I found the correct solution and as usual it was easier than I thought, so I was basically trying to decompress a page that's not gzipped for that reason it was failing. Now once I hit the second page (which is compressed) I get a gzipped response where the code above should handle it. Also if anyone wants the solution I used a modified interceptor just like the one in this answer so you don't need to use a custom function to handle the decompression.
我修改了unzip
方法,以使okhttp interceptor
可以处理压缩和未压缩的响应:
I modified the unzip
method to make the okhttp interceptor
work with compressed and uncompressed responses:
OkHttpClient.Builder clientBuilder = new OkHttpClient.Builder().addInterceptor(new UnzippingInterceptor());
OkHttpClient client = clientBuilder.build();
拦截器就像dis:
private class UnzippingInterceptor implements Interceptor {
@Override
public Response intercept(Chain chain) throws IOException {
Response response = chain.proceed(chain.request());
return unzip(response);
// copied from okhttp3.internal.http.HttpEngine (because is private)
private Response unzip(final Response response) throws IOException
{
if (response.body() == null)
{
return response;
}
//check if we have gzip response
String contentEncoding = response.headers().get("Content-Encoding");
//this is used to decompress gzipped responses
if (contentEncoding != null && contentEncoding.equals("gzip"))
{
Long contentLength = response.body().contentLength();
GzipSource responseBody = new GzipSource(response.body().source());
Headers strippedHeaders = response.headers().newBuilder().build();
return response.newBuilder().headers(strippedHeaders)
.body(new RealResponseBody(response.body().contentType().toString(), contentLength, Okio.buffer(responseBody)))
.build();
}
else
{
return response;
}
}
}
}
这篇关于okhttp 3:如何使用Java/Android手动解压缩gzip/deflate响应的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!