使用Apache Async Http客户端进行异步响应流式传输 [英] Async response streaming with Apache Async Http client

问题描述

我正在使用apache异步http客户端从azure存储流式传输对象.

I am using apache async http client to stream objects from azure storage.

我只需要返回具有关联流的HttpResponse对象.我的客户实际上必须从该流中读取内容以将文件存储在本地.

I only need to return the HttpResponse object which has the stream associated. My clients will actually have to read from that stream to store the file locally.

因此，Apache Async客户端使用BasicAsyncResponseConsumer实际上在调用完成的回调之前将整个文件缓冲在本地内存中.

So Apache Async clients use a BasicAsyncResponseConsumer which actually buffers the entire file in local memory before calling the completed callback.

我正在尝试创建自己的AbstractAsyncResponseConsumer实现，这样我就可以流式传输响应主体，而不是先实际存储它，但直到现在都没有成功.

I am trying to create my own implementation of AbstractAsyncResponseConsumer so that I can stream the response body instead of actually storing it first but have been unsuccessful to do so till now.

这是供参考的裸机消费类->

Here is the bare bones cosumer class for reference ->

public class MyConsumer extends` AbstractAsyncResponseConsumer<HttpResponse> {
@Override
protected void onResponseReceived(HttpResponse response) throws HttpException, IOException {

}

@Override
protected void onContentReceived(ContentDecoder decoder, IOControl ioctrl) throws IOException {

}

@Override
protected void onEntityEnclosed(HttpEntity entity, ContentType contentType) throws IOException {

}

@Override
protected HttpResponse buildResult(HttpContext context) throws Exception {
    return null;
}

@Override
protected void releaseResources() {

}

}

这是发送请求并返回响应的代码->

And here is the code to send the request and return the response ->

public void getFile(HttpRequestBase request) {

    MyConsumer myConsumer = new MyConsumer();
    HttpAsyncRequestProducer producer = 
    HttpAsyncMethods.create(request);
    CompletableFuture<HttpResponse> future = new CompletableFuture<>();
    return Future<HttpResponse> responseFuture = 
    httpclient.execute(producer,myConsumer,                                                                                                                   
    new FutureCallback<HttpResponse>() {
      @Override
      public void completed(HttpResponse result) {
     //This is called only when all the response body has been read
     //future.complete(Result)

      }
      @Override                                                                      
      public void failed(Exception ex) {
      }
      @Override
      public void cancelled() {                                                                       
      }
   });

return future;

 }

我将向我的客户端返回HttpResponse对象的CompletableFuture.

I will be returning a CompletableFuture of the HttpResponse object to my clients.

他们不应该等待我的http客户端首先读取本地缓冲区中的所有响应正文.

They shouldnt be waiting for my http client to read all the response body first in local buffer.

理想情况下，它们应该直接从响应对象中提供的流开始复制.

They ideally should start copying directly from the stream provided in the response object.

我应该在消费者的实现中添加些什么以获得理想的结果?

What should I add inmy implementation of the consumer to get the desired result ?

推荐答案

我不知道您是否仍然遇到此问题，但是如果您想要的是实际上流传输数据的InputStream，那么您将需要使用阻止版本的Apache HttpClient.

I don't know if you still have this problem, but if what you want is an InputStream that actually streams data, then you'll want to use the blocking version of Apache HttpClient.

Java的内置InputStream和OutputStream本质上是阻塞的，因此返回CompletableFuture的InputStream本质上无法达到目的. BasicAsyncResponseConsumer将整个响应缓冲在内存中实际上是正确的做法，因为这是使其真正无阻塞的唯一方法.

Java's built-in InputStream and OutputStream are inherently blocking, so returning a CompletableFuture of InputStream essentially defeats the purpose. BasicAsyncResponseConsumer buffering the entire response in memory is actually the right thing to do, because that's the only way of making it truly non-blocking.

您可以查看的另一个选项是HttpAsyncMethods.createZeroCopyConsumer.它的作用是以完全无阻塞的方式将内容存储到文件中. 这是一个示例:

Another option you can take a look at is HttpAsyncMethods.createZeroCopyConsumer. What it does is that it stores the content to a file in a completely non-blocking way. Here's an example:

        try (CloseableHttpAsyncClient client = HttpAsyncClients.createDefault()) {
            client.start();
            final CompletableFuture<HttpResponse> cf = new CompletableFuture<>();
            client.execute(
                    HttpAsyncMethods.createGet("https://example.com"),
                    HttpAsyncMethods.createZeroCopyConsumer(new File("foo.html")),
                    new FutureCallback<HttpResponse>() {
                        @Override
                        public void completed(HttpResponse result) {
                            cf.complete(result);
                        }
                        @Override
                        public void failed(Exception ex) {
                            cf.completeExceptionally(ex);
                        }
                        @Override
                        public void cancelled() {
                            cf.cancel(true);
                        }
                    });
            // When cf completes, the file will be ready.
            // The InputStream inside the HttpResponse will be the FileInputStream of the created file.
        }

这篇关于使用Apache Async Http客户端进行异步响应流式传输的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！