如何设置InputStream内容的长度 [英] How to set InputStream content Length

问题描述

我正在将文件上传到Amazon S3存储桶.文件正在上传，但我收到以下警告.

I am uploading files to Amazon S3 bucket. The files are being uploaded but i get the following Warning.

警告:未为流数据指定内容长度.流内容 将被缓冲在内存中，并可能导致内存不足错误.

WARNING: No content length specified for stream data. Stream contents will be buffered in memory and could result in out of memory errors.

所以我在代码中添加了以下行

So I added the following line to my code

metaData.setContentLength(IOUtils.toByteArray(input).length);

但随后我收到以下消息.我什至不知道这是警告还是什么.

but then i got the following message. I don't even know if it is a warning or what.

读取的数据长度与预期的长度不同:dataLength = 0; 预期长度= 111992; includeSkipped = false; in.getClass()= class sun.net.httpserver.FixedLengthInputStream; markedSupported = false; 标记= 0; resetSinceLastMarked = false; markCount = 0; resetCount = 0

Data read has a different length than the expected: dataLength=0; expectedLength=111992; includeSkipped=false; in.getClass()=class sun.net.httpserver.FixedLengthInputStream; markedSupported=false; marked=0; resetSinceLastMarked=false; markCount=0; resetCount=0

如何将contentLength设置为InputSteam的metaData?任何帮助将不胜感激.

How can i set contentLength to the metaData of InputSteam? Any help would be greatly appreciated.

推荐答案

使用IOUtils.toByteArray读取数据时，将消耗InputStream.当AWS API尝试读取它时，它的长度为零.

When you read the data with IOUtils.toByteArray, this consumes the InputStream. When the AWS API tries to read it, it's zero length.

将内容读取到字节数组中，并提供将该数组包装到API的InputStream:

Read the contents into a byte array and provide an InputStream wrapping that array to the API:

byte[] bytes = IOUtils.toByteArray(input);
metaData.setContentLength(bytes.length);
ByteArrayInputStream byteArrayInputStream = new ByteArrayInputStream(bytes);
PutObjectRequest putObjectRequest = new PutObjectRequest(bucket, key, byteArrayInputStream, metadata);
client.putObject(putObjectRequest);

您应该考虑使用分段上传API，以避免将整个InputStream加载到内存中.例如:

You should consider using the multipart upload API to avoid loading the whole InputStream into memory. For example:

byte[] bytes = new byte[BUFFER_SIZE];
String uploadId = client.initiateMultipartUpload(new InitiateMultipartUploadRequest(bucket, key)).getUploadId();

int bytesRead = 0;
int partNumber = 1;
List<UploadPartResult> results = new ArrayList<>();
bytesRead = input.read(bytes);
while (bytesRead >= 0) {
    UploadPartRequest part = new UploadPartRequest()
        .withBucketName(bucket)
        .withKey(key)
        .withUploadId(uploadId)
        .withPartNumber(partNumber)
        .withInputStream(new ByteArrayInputStream(bytes, 0, bytesRead))
        .withPartSize(bytesRead);
    results.add(client.uploadPart(part));
    bytesRead = input.read(bytes);
    partNumber++;
}
CompleteMultipartUploadRequest completeRequest = new CompleteMultipartUploadRequest()
    .withBucketName(bucket)
    .withKey(key)
    .withUploadId(uploadId)
    .withPartETags(results);
client.completeMultipartUpload(completeRequest);

这篇关于如何设置InputStream内容的长度的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！