带有 InputStream 长度示例的 AmazonS3 putObject [英] AmazonS3 putObject with InputStream length example

问题描述

我正在使用 Java 将文件上传到 S3 - 这是我目前得到的:

AmazonS3 s3 = new AmazonS3Client(new BasicAWSCredentials("XX","YY"));列表<桶>桶 = s3.listBuckets();s3.putObject(new PutObjectRequest(buckets.get(0).getName(), fileName, stream, new ObjectMetadata()));

文件正在上传，但当我没有设置内容长度时出现警告:

<块引用>

com.amazonaws.services.s3.AmazonS3Client putObject:没有为流指定内容长度>数据.流内容将缓冲在内存中，并可能导致内存不足错误.

这是我正在上传的文件，stream 变量是一个 InputStream，我可以从中获取这样的字节数组:IOUtils.toByteArray(流).

因此，当我尝试设置内容长度和 MD5(取自 此处) 像这样:

//获取 MD5 base64 哈希值MessageDigest messageDigest = MessageDigest.getInstance("MD5");messageDigest.reset();messageDigest.update(IOUtils.toByteArray(stream));byte[] resultByte = messageDigest.digest();String hashtext = new String(Hex.encodeHex(resultByte));ObjectMetadata meta = new ObjectMetadata();meta.setContentLength(IOUtils.toByteArray(stream).length);meta.setContentMD5(hashtext);

它会导致从 S3 返回以下错误:

<块引用>

您指定的 Content-MD5 无效.

我做错了什么?

感谢任何帮助！

PS 我使用的是 Google App Engine - 我无法将文件写入磁盘或 创建临时文件，因为 AppEngine 不支持 FileOutputStream.

解决方案

因为原来的问题一直没有人回答，我也遇到了同样的问题，MD5 问题的解决方案是 S3 不想要 Hex我们通常会想到的编码的 MD5 字符串.

相反，我不得不这样做.

//内容是传入的 InputStreambyte[] resultByte = DigestUtils.md5(content);String streamMD5 = new String(Base64.encodeBase64(resultByte));metaData.setContentMD5(streamMD5);

基本上他们想要的 MD5 值是 Base64 编码的原始 MD5 字节数组，而不是十六进制字符串.当我切换到这个时，它开始对我很有用.

I am uploading a file to S3 using Java - this is what I got so far:

AmazonS3 s3 = new AmazonS3Client(new BasicAWSCredentials("XX","YY"));

List<Bucket> buckets = s3.listBuckets();

s3.putObject(new PutObjectRequest(buckets.get(0).getName(), fileName, stream, new ObjectMetadata()));

The file is being uploaded but a WARNING is raised when I am not setting the content length:

com.amazonaws.services.s3.AmazonS3Client putObject: No content length specified for stream > data.  Stream contents will be buffered in memory and could result in out of memory errors.

This is a file I am uploading and the stream variable is an InputStream, from which I can get the byte array like this: IOUtils.toByteArray(stream).

So when I try to set the content length and MD5 (taken from here) like this:

// get MD5 base64 hash
MessageDigest messageDigest = MessageDigest.getInstance("MD5");
messageDigest.reset();
messageDigest.update(IOUtils.toByteArray(stream));
byte[] resultByte = messageDigest.digest();
String hashtext = new String(Hex.encodeHex(resultByte));

ObjectMetadata meta = new ObjectMetadata();
meta.setContentLength(IOUtils.toByteArray(stream).length);
meta.setContentMD5(hashtext);

It causes the following error to come back from S3:

The Content-MD5 you specified was invalid.

What am I doing wrong?

Any help appreciated!

P.S. I am on Google App Engine - I cannot write the file to disk or create a temp file because AppEngine does not support FileOutputStream.

解决方案

Because the original question was never answered, and I had to run into this same problem, the solution for the MD5 problem is that S3 doesn't want the Hex encoded MD5 string we normally think about.

Instead, I had to do this.

// content is a passed in InputStream
byte[] resultByte = DigestUtils.md5(content);
String streamMD5 = new String(Base64.encodeBase64(resultByte));
metaData.setContentMD5(streamMD5);

Essentially what they want for the MD5 value is the Base64 encoded raw MD5 byte-array, not the Hex string. When I switched to this it started working great for me.

这篇关于带有 InputStream 长度示例的 AmazonS3 putObject的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！