发送HTTP时的OutputStream OutOfMemoryError [英] OutputStream OutOfMemoryError when sending HTTP

问题描述

我正在尝试将大型视频/图像文件从本地文件系统发布到http路径，但是在一段时间后我遇到内存不足错误...

I am trying to publish a large video/image file from the local file system to an http path, but I run into an out of memory error after some time...

这里是代码

public boolean publishFile(URI publishTo, String localPath) throws Exception {
    InputStream istream = null;
    OutputStream ostream = null;
    boolean isPublishSuccess = false;

    URL url = makeURL(publishTo.getHost(), this.port, publishTo.getPath());
    HttpURLConnection conn = (HttpURLConnection) url.openConnection();


    if (conn != null) {

        try {

            conn.setDoOutput(true);
            conn.setDoInput(true);
            conn.setRequestMethod("PUT");
            istream = new FileInputStream(localPath);
            ostream = conn.getOutputStream();

            int n;
            byte[] buf = new byte[4096];
            while ((n = istream.read(buf, 0, buf.length)) > 0) {
                ostream.write(buf, 0, n); //<--- ERROR happens on this line.......???
            }

            int rc = conn.getResponseCode();

            if (rc == 201) {
                isPublishSuccess = true;
            }

        } catch (Exception ex) {
            log.error(ex);
        } finally {
            if (ostream != null) {
                ostream.close();
            }

            if (istream != null) {
                istream.close();
            }
        }
    }

    return isPublishSuccess;

}

HEre是我得到的错误...

HEre is the error i am getting...

Exception in thread "Thread-8773" java.lang.OutOfMemoryError: Java heap space
    at java.util.Arrays.copyOf(Arrays.java:2786)
    at java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:94)
    at sun.net.www.http.PosterOutputStream.write(PosterOutputStream.java:61)
    at com.test.HTTPClient.publishFile(HTTPClient.java:110)
    at com.test.HttpFileTransport.put(HttpFileTransport.java:97)

推荐答案

HttpUrlConnection 正在缓冲数据，以便它可以设置 Content-Length 标题（每 HTTP规范）。

The HttpUrlConnection is buffering the data so that it can set the Content-Length header (per HTTP spec).

如果您的目标服务器支持，则另一种方法是使用 chunked 转移。这将一次仅缓冲一小部分数据。但是，并非所有服务都支持它（例如，亚马逊S3没有）。

One alternative, if your destination server supports it, is to use "chunked" transfers. This will buffer only a small portion of data at a time. However, not all services support it (Amazon S3, for example, doesn't).

另一个替代方案（并且更好的方法）是使用 Jakarta HttpClient 。您可以在文件的请求中设置实体，连接代码将适当地设置请求标头。

Another alternative (and imo a better one) is to use Jakarta HttpClient. You can set the "entity" in a request from a file, and the connection code will set request headers appropriately.

编辑： nos 评论说OP可以调用 HttpURLConnection.setFixedLengthStreamingMode（long length）。我没有意识到这种方法;它是在1.5中添加的，从那以后我没有使用过这个类。

nos commented that the OP could call HttpURLConnection.setFixedLengthStreamingMode(long length). I was unaware of this method; it was added in 1.5, and I haven't used this class since then.

然而，我仍建议使用Jakarta HttpClient，原因很简单，它减少了OP必须维护的代码量。代码是样板文件，但仍有可能出错：

However, I still suggest using Jakarta HttpClient, for the simple reason that it reduces the amount of code that the OP has to maintain. Code that is boilerplate, yet still has the potential for errors:

• OP正确处理要在输入和输出之间复制的循环。通常当我看到这样的示例时，海报要么没有正确检查返回的缓冲区大小，要么继续重新分配缓冲区。恭喜，但你现在必须确保你的继任者尽可能多的关心。


• 异常处理并不是那么好。是的，OP记得关闭 finally 块中的连接，并再次祝贺。除了 close（）调用之一可能抛出 IOException ，保持另一个执行。整个方法抛出 Exception ，这样编译器就不会帮助捕获类似的错误。


• 我算31用于设置和执行响应的代码行（不包括响应代码检查和URL计算，但包括try / catch / finally）。使用HttpClient，这将在半打LOC的范围内。

• The OP correctly handles the loop to copy between input and output. Usually when I see an example of this, the poster either doesn't properly check the returned buffer size, or keeps re-allocating the buffers. Congratulations, but you now have to ensure that your successors take as much care.
• The exception handling isn't quite so good. Yes, the OP remembers to close the connections in a finally block, and again, congratulations on that. Except that either of the close() calls could throw IOException, keeping the other from executing. And the method as a whole throws Exception, so that the compiler isn't going to help catch similar errors.
• I count 31 lines of code to setup and execute the response (excluding the response code check and the URL computation, but including the try/catch/finally). With HttpClient, this would be somewhere in the range of a half dozen LOC.

即使OP已完美地编写此代码，将其重构为类似于Jakarta Commons IO中的方法， s /他不应该这样做。此代码已由其他人编写和测试。我知道浪费我的时间来重写它，并怀疑它也浪费了OP的时间。

Even if the OP had written this code perfectly, and refactored it into methods similar to those in Jakarta Commons IO, s/he shouldn't do that. This code has been written and tested by others. I know that it's a waste of my time to rewrite it, and suspect that it's a waste of the OP's time as well.

这篇关于发送HTTP时的OutputStream OutOfMemoryError的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！