Java Heap Space不足以在AWS S3上上传文件 [英] Java Heap Space is insufficient to upload files on AWS S3

问题描述

我正在尝试使用Java-AWS API在AWS S3上上传文件.问题是我的应用程序无法上传大型文件，因为堆已达到极限. 错误:java.lang.OutOfMemoryError:Java堆空间

I'm trying to upload a file on AWS S3 by using Java-AWS API. The problem is my application is unable to upload large sized files because the heap is reaching its limit. Error: java.lang.OutOfMemoryError: Java heap space

我个人认为扩展堆内存不是永久性的解决方案，因为我必须上传最大100 GB的文件.我该怎么办?

I personally think extending heap memory isn't a permanent solution because I have to upload file upto 100 gb. What should I do ?

这是代码段:

        BasicAWSCredentials awsCreds = new BasicAWSCredentials(AID, Akey);
        AmazonS3 s3Client = AmazonS3ClientBuilder.standard()
        .withRegion(Regions.fromName("us-east-2"))
        .withCredentials(new AWSStaticCredentialsProvider(awsCreds))
        .build();

        InputStream Is=file.getInputStream();

        boolean buckflag = s3Client.doesBucketExist(ABuck);
        if(buckflag != true){
           s3Client.createBucket(ABuck);
        }
        s3Client.putObject(new PutObjectRequest(ABuck, AFkey,file.getInputStream(),new ObjectMetadata() ).withCannedAcl(CannedAccessControlList.PublicRead));

推荐答案

我强烈建议在ObjectMetadata上的setContentLength()，因为:

I strongly recommend to setContentLength() on ObjectMetadata, since:

..如果未提供，则库将必须缓冲以计算输入流的内容.

..If not provided, the library will have to buffer the contents of the input stream in order to calculate it.

(..这可能会导致足够大"的文件上出现OutOfMemory.)

(..which predictably will lead to OutOfMemory on "sufficient large" files.)

源: PutObjectRequest javadoc

应用于您的代码:

 // ...
 ObjectMetadata omd = new ObjectMetadata();
 // a tiny code line, but with a "huge" information gain and memory saving!;)
 omd.setContentLength(file.length());

 s3Client.putObject(new PutObjectRequest(ABuck, AFkey, file.getInputStream(), omd).withCannedAcl(CannedAccessControlList.PublicRead));
 // ...

这篇关于Java Heap Space不足以在AWS S3上上传文件的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！