s3.getObject()。createReadStream():如何捕获错误? [英] s3.getObject().createReadStream() : How to catch the error?
问题描述
我正在尝试编写程序从s3获取zip文件,解压缩,然后将其上传到S3。
但我发现了两个我无法捕捉到的例外情况。
I am trying to write a program to get a zip file from s3, unzip it, then upload it to S3. But I found two exceptions that I can not catch.
1. StreamContentLengthMismatch:流内容长度不匹配。收到980323883,共5770104761字节。
这种情况不规律地发生。
1. StreamContentLengthMismatch: Stream content length mismatch. Received 980323883 of 5770104761 bytes.
This occurs irregularly.
2。 NoSuchKey:The指定的密钥不存在。
当我输入错误的密钥时会发生这种情况。
2. NoSuchKey: The specified key does not exist.
This happens when I input the wrong key.
当发生这两个异常时,此程序崩溃。
When these two exceptions occur, this program crashes.
我想正确地捕捉并处理这两个例外。
I'd like to catch and handle these two exceptions correctly.
我想防止崩溃。
const unzipUpload = () => {
return new Promise((resolve, reject) => {
let rStream = s3.getObject({Bucket: 'bucket', Key: 'hoge/hoge.zip'})
.createReadStream()
.pipe(unzip.Parse())
.on('entry', function (entry) {
if(entry.path.match(/__MACOSX/) == null){
// pause
if(currentFileCount - uploadedFileCount > 10) rStream.pause()
currentFileCount += 1
var fileName = entry.path;
let up = entry.pipe(uploadFromStream(s3,fileName))
up.on('uploaded', e => {
uploadedFileCount += 1
console.log(currentFileCount, uploadedFileCount)
//resume
if(currentFileCount - uploadedFileCount <= 10) rStream.resume()
if(uploadedFileCount === allFileCount) resolve()
entry.autodrain()
}).on('error', e => {
reject()
})
}
}).on('error', e => {
console.log("unzip error")
reject()
}).on('finish', e => {
allFileCount = currentFileCount
})
rStream.on('error', e=> {
console.log(e)
reject(e)
})
})
}
function uploadFromStream(s3,fileName) {
var pass = new stream.PassThrough();
var params = {Bucket: "bucket", Key: "hoge/unzip/" + fileName, Body: pass};
let request = s3.upload(params, function(err, data) {
if(err) pass.emit('error')
if(!err) pass.emit('uploaded')
})
request.on('httpUploadProgress', progress => {
console.log(progress)
})
return pass
}
这是解压缩时使用的库。
https://github.com/mhr3/unzip-stream
This is the library I use when unzipping. https://github.com/mhr3/unzip-stream
帮助我!!
推荐答案
如果你想抓住 NoSuchKey
createReadStream抛出的错误
你有两个选择:
If you'd like to catch the NoSuchKey
error thrown by createReadStream
you have 2 options:
- 在阅读之前检查密钥是否存在。
- 从流中捕获错误
第一个:
s3.getObjectMetadata(key)
.promise()
.then(() => {
// This will not throw error anymore
s3.getObject().createReadStream();
})
.catch(error => {
if (error.statusCode === 404) {
// Catching NoSuchKey
}
});
唯一一种情况,如果文件在瞬间被删除,在解析之间就不会发现错误响应来自 getObjectMetadata
并运行 createReadStream
The only case when you won't catch error if file was deleted in a split second, between parsing response from getObjectMetadata
and running createReadStream
第二个:
s3.getObject().createReadStream().on('error', error => {
// Catching NoSuchKey & StreamContentLengthMismatch
});
这是一种更通用的方法,可以捕获所有其他错误,例如网络问题。
This is a more generic approach and will catch all other errors, like network problems.
这篇关于s3.getObject()。createReadStream():如何捕获错误?的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!