如何使用Java获取上传到Amazon S3的文件的进度状态 [英] How to get the progress status of the file uploaded to Amazon S3 using Java
问题描述
我正在使用Java将多个文件上传到Amazon S3.
I'm uploading multiple files to Amazon S3 using Java.
我正在使用的代码如下:
The code I'm using is as follows:
MultipartHttpServletRequest multipartRequest = (MultipartHttpServletRequest) request;
MultiValueMap < String,
MultipartFile > map = multipartRequest.getMultiFileMap();
try {
if (map != null) {
for (String filename: map.keySet()) {
List < MultipartFile > fileList = map.get(filename);
incrPercentge = 100 / fileList.size();
request.getSession().setAttribute("incrPercentge", incrPercentge);
for (MultipartFile mpf: fileList) {
/*
* custom input stream wrap to original input stream to get
* the progress
*/
ProgressInputStream inputStream = new ProgressInputStream("test", mpf.getInputStream(), mpf.getBytes().length);
ObjectMetadata metadata = new ObjectMetadata();
metadata.setContentType(mpf.getContentType());
String key = Util.getLoginUserName() + "/" + mpf.getOriginalFilename();
PutObjectRequest putObjectRequest = new PutObjectRequest(
Constants.S3_BUCKET_NAME, key, inputStream, metadata).withStorageClass(StorageClass.ReducedRedundancy);
PutObjectResult response = s3Client.putObject(putObjectRequest);
}
}
}
} catch(Exception e) {
e.printStackTrace();
}
我必须创建自定义输入流才能获取Amazon S3消耗的数字字节.我是从这里的问题中得到这个想法的:上传文件或带有进度回调的InputStream到S3
I have to create the custom input stream to get the number byte consumed by Amazon S3. I got that idea from the question here: Upload file or InputStream to S3 with a progress callback
我的ProgressInputStream类代码如下:
package com.spectralnetworks.net.util;
import java.io.IOException;
import java.io.InputStream;
import org.apache.commons.vfs.FileContent;
import org.apache.commons.vfs.FileSystemException;
public class ProgressInputStream extends InputStream {
private final long size;
private long progress,
lastUpdate = 0;
private final InputStream inputStream;
private final String name;
private boolean closed = false;
public ProgressInputStream(String name, InputStream inputStream, long size) {
this.size = size;
this.inputStream = inputStream;
this.name = name;
}
public ProgressInputStream(String name, FileContent content)
throws FileSystemException {
this.size = content.getSize();
this.name = name;
this.inputStream = content.getInputStream();
}
@Override
public void close() throws IOException {
super.close();
if (closed) throw new IOException("already closed");
closed = true;
}
@Override
public int read() throws IOException {
int count = inputStream.read();
if (count > 0) progress += count;
lastUpdate = maybeUpdateDisplay(name, progress, lastUpdate, size);
return count;
}@Override
public int read(byte[] b, int off, int len) throws IOException {
int count = inputStream.read(b, off, len);
if (count > 0) progress += count;
lastUpdate = maybeUpdateDisplay(name, progress, lastUpdate, size);
return count;
}
/**
* This is on reserach to show a progress bar
* @param name
* @param progress
* @param lastUpdate
* @param size
* @return
*/
static long maybeUpdateDisplay(String name, long progress, long lastUpdate, long size) {
/* if (Config.isInUnitTests()) return lastUpdate;
if (size < B_IN_MB/10) return lastUpdate;
if (progress - lastUpdate > 1024 * 10) {
lastUpdate = progress;
int hashes = (int) (((double)progress / (double)size) * 40);
if (hashes > 40) hashes = 40;
String bar = StringUtils.repeat("#",
hashes);
bar = StringUtils.rightPad(bar, 40);
System.out.format("%s [%s] %.2fMB/%.2fMB\r",
name, bar, progress / B_IN_MB, size / B_IN_MB);
System.out.flush();
}*/
System.out.println("name " + name + " progress " + progress + " lastUpdate " + lastUpdate + " " + "sie " + size);
return lastUpdate;
}
}
但这无法正常工作.它会立即打印直至文件大小，如下所示:
But this is not working properly. It is printing immediately up to the file size as follows:
name test progress 4096 lastUpdate 0 sie 30489
name test progress 8192 lastUpdate 0 sie 30489
name test progress 12288 lastUpdate 0 sie 30489
name test progress 16384 lastUpdate 0 sie 30489
name test progress 20480 lastUpdate 0 sie 30489
name test progress 24576 lastUpdate 0 sie 30489
name test progress 28672 lastUpdate 0 sie 30489
name test progress 30489 lastUpdate 0 sie 30489
name test progress 30489 lastUpdate 0 sie 30489
并且实际的上载要花费更多的时间(在打印行后要花费10倍以上的时间).
And the actual uploading is taking more time (more than 10 times after printing the lines).
我应该怎么做才能获得真实的上传状态?
What I should do so that I can get a true upload status?
推荐答案
我使用下面的代码以最佳的方式获得了真正的进度状态，这是我的问题的答案
I got the answer of my questions the best way get the true progress status by using below code
ObjectMetadata metadata = new ObjectMetadata();
metadata.setContentType(mpf.getContentType());
String key = Util.getLoginUserName() + "/"
+ mpf.getOriginalFilename();
metadata.setContentLength(mpf.getSize());
PutObjectRequest putObjectRequest = new PutObjectRequest(
Constants.S3_BUCKET_NAME, key, mpf.getInputStream(),
metadata)
.withStorageClass(StorageClass.ReducedRedundancy);
putObjectRequest.setProgressListener(new ProgressListener() {
@Override
public void progressChanged(ProgressEvent progressEvent) {
System.out.println(progressEvent
.getBytesTransfered()
+ ">> Number of byte transfered "
+ new Date());
progressEvent.getBytesTransfered();
double totalByteRead = request
.getSession().getAttribute(
Constants.TOTAL_BYTE_READ) != null ? (Double) request
.getSession().getAttribute(Constants.TOTAL_BYTE_READ) : 0;
totalByteRead += progressEvent.getBytesTransfered();
request.getSession().setAttribute(Constants.TOTAL_BYTE_READ, totalByteRead);
System.out.println("total Byte read "+ totalByteRead);
request.getSession().setAttribute(Constants.TOTAL_PROGRESS, (totalByteRead/size)*100);
System.out.println("percentage completed >>>"+ (totalByteRead/size)*100);
if (progressEvent.getEventCode() == ProgressEvent.COMPLETED_EVENT_CODE) {
System.out.println("completed ******");
}
}
});
s3Client.putObject(putObjectRequest);
我之前的代码的问题是，我没有在元数据中设置内容长度，所以我没有得到真正的进度状态.下面的行是从PutObjectRequest类API复制的
The problem with my previous code was , I was not setting the content length in meta data so i was not getting the true progress status. The below line is copy from PutObjectRequest class API
构造一个新的PutObjectRequest对象，以将数据流上载到指定的存储桶和键.构造请求之后，用户可以选择指定对象元数据或罐头ACL.
Constructs a new PutObjectRequest object to upload a stream of data to the specified bucket and key. After constructing the request, users may optionally specify object metadata or a canned ACL as well.
必须在对象元数据参数中指定数据流的内容长度； Amazon S3要求在上传数据之前先将其传递.不指定内容长度将导致输入流的全部内容在内存中本地缓存，从而可以计算内容长度，这可能导致负面的性能问题.
Content length for the data stream must be specified in the object metadata parameter; Amazon S3 requires it be passed in before the data is uploaded. Failure to specify a content length will cause the entire contents of the input stream to be buffered locally in memory so that the content length can be calculated, which can result in negative performance problems.
这篇关于如何使用Java获取上传到Amazon S3的文件的进度状态的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
