在CXF jax-rs中禁用多部分缓存 [英] Disabling Multipart Caching in CXF jax-rs

问题描述

我

I posted this question to the CXF list, without any luck. So here we go. I am trying to upload large files to a remote server (think of them virtual machine disks). So I have a restful service that accepts upload requests. The handler for the upload looks like:

@POST
@Consumes(MediaType.MULTIPART_FORM_DATA)
@Path("/doupload")
public Response receiveStream(MultipartBody multipart) {
    List<Attachment> allAttachments = body.getAllAttachments();
    Attachment att = null;
    for (Attachment b : allAttachments) {
        if (UPLOAD_FILE_DESCRIPTOR.equals(b.getContentId())) {
            att = b;
        }
    }
    Assert.notNull(att);
    DataHandler dh = att.getDataHandler();
    if (dh == null) {
        throw new WebApplicationException(HTTP_BAD_REQUEST);
    }
    try {
        InputStream is = dh.getInputStream();
        byte[] buf = new byte[65536];
        int n;
        OutputStream os = getOutputStream();
        while ((n = is.read(buf)) > 0) {
            os.write(buf, 0, n);
        }
        ResponseBuilder rb = Response.status(HTTP_CREATED);
        return rb.build();
    } catch (IOException e) {
        log.error("Got exception=", e);
        throw new WebApplicationException(HTTP_INTERNAL_ERROR);
    } catch (NoSuchAlgorithmException e) {
        log.error("Got exception=", e);
        throw new WebApplicationException(HTTP_INTERNAL_ERROR);
    } finally {}

}

此代码的客户端非常简单:

The client for this code is fairly simple:

public void sendLargeFile(String filename) {
    WebClient wc = WebClient.create(targetUrl);
    InputStream is = new FileInputStream(new File(filename));
    Response r = wc.post(new Attachment(Constants.UPLOAD_FILE_DESCRIPTOR,
        MediaType.APPLICATION_OCTET_STREAM, is));
}

代码在功能方面运行良好.在性能方面，我注意到在我的处理程序(receiveStream()方法)从流中获取第一个字节之前，实际上整个流都被持久化到了一个临时文件中(使用CachedOutputStream).不幸的是，出于我的目的，这是不可接受的.

The code works fine in terms of functionality. In terms of performance, I noticed that before my handler (receiveStream() method) gets the first byte out of the stream, the whole stream actually gets persisted into a temporary file (using a CachedOutputStream). Unfortunately, this is not acceptable for my purposes.

• 我的处理程序只是将传入的字节传递到后端存储系统(虚拟机磁盘存储库)，而等待将整个磁盘写入高速缓存以仅再次读取需要很多时间，从而占用了很多时间资源，并降低吞吐量.
• 由于该应用程序在云中运行，因此写入和重新读取这些块会产生一定的成本，并且云提供商会按读取/写入的每个块收费.
• 由于每个字节都写入本地磁盘，因此我的服务VM必须具有足够的磁盘空间来容纳所有正在上载的流的总大小(即，如果我有10个100GB的上载，则我必须有1TB的磁盘只是为了缓存内容).这又是一笔额外的钱，因为服务VM的规模急剧增长，并且云提供商也为配置的磁盘大小收费.

鉴于所有这些，我正在寻找一种使用HTTP InputStream(或尽可能接近它)从那里直接读取附件并随后对其进行处理的方法.我想这个问题会转换为以下其中一项: -有没有办法告诉CXF不做缓存 -或-有没有一种方法可以通过CXF输出流(我编写的一个)来使用，而不是使用CachedOutputStream

Given all of this, I am looking for a way to use the HTTP InputStream (or as close to it as possible) to read the attachment directly from there and handle it afterwards. I guess the question translates into one of: - Is there a way to tell CXF not do caching - OR - is there a way to pass CXF an output stream (one I write) to use, rather than using CachedOutputStream

我发现了类似的问题

I found a similar question here. The resolution says use CXF 2.2.3 or later, I am using 2.4.4 (and tried with 2.7.0) with no luck.

谢谢.

推荐答案

我认为从逻辑上讲是不可能的(在CXF或其他任何地方).您正在调用getAllAttachements()，这意味着服务器应该从HTTP输入流中收集有关它们的信息.这意味着整个流必须进入内存以进行 MIME 解析.

I think it's logically not possible (neither in CXF or anywhere else). You're calling getAllAttachements(), which means that the server should collect information about them from the HTTP input stream. It means that the entire stream has to go into memory for MIME parsing.

在这种情况下，您应该直接使用流，并自行进行MIME解析:

In your case you should work directly with the stream, and do the MIME parsing yourself:

public Response receiveStream(InputStream input) {

现在，您可以完全控制输入，并且可以逐字节将其消耗到内存中.

Now you have full control of the input and can consume it into memory byte-by-byte.

这篇关于在CXF jax-rs中禁用多部分缓存的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！