s3.getObject()。createReadStream():如何捕获错误? [英] s3.getObject().createReadStream() : How to catch the error?

查看:553
本文介绍了s3.getObject()。createReadStream():如何捕获错误?的处理方法,对大家解决问题具有一定的参考价值,需要的朋友们下面随着小编来一起学习吧!

问题描述

我正在尝试编写程序从s3获取zip文件,解压缩,然后将其上传到S3。
但我发现了两个我无法捕捉到的例外情况。

I am trying to write a program to get a zip file from s3, unzip it, then upload it to S3. But I found two exceptions that I can not catch.

1. StreamContentLengthMismatch:流内容长度不匹配。收到980323883,共5770104761字节。这种情况不规律地发生。

1. StreamContentLengthMismatch: Stream content length mismatch. Received 980323883 of 5770104761 bytes. This occurs irregularly.

2。 NoSuchKey:The指定的密钥不存在。当我输入错误的密钥时会发生这种情况。

2. NoSuchKey: The specified key does not exist. This happens when I input the wrong key.

当发生这两个异常时,此程序崩溃。

When these two exceptions occur, this program crashes.

我想正确地捕捉并处理这两个例外。

I'd like to catch and handle these two exceptions correctly.

我想防止崩溃。

   const unzipUpload = () => {
        return new Promise((resolve, reject) => {
            let rStream = s3.getObject({Bucket: 'bucket', Key: 'hoge/hoge.zip'})
                .createReadStream()
                    .pipe(unzip.Parse())
                    .on('entry', function (entry) {
                        if(entry.path.match(/__MACOSX/) == null){

                            // pause
                            if(currentFileCount - uploadedFileCount > 10) rStream.pause()

                            currentFileCount += 1
                            var fileName = entry.path;
                            let up = entry.pipe(uploadFromStream(s3,fileName))

                            up.on('uploaded', e => {
                                uploadedFileCount += 1
                                console.log(currentFileCount, uploadedFileCount)

                                //resume
                                if(currentFileCount - uploadedFileCount <= 10) rStream.resume()

                                if(uploadedFileCount === allFileCount) resolve()
                                entry.autodrain()
                            }).on('error', e => {
                                reject()
                            })
                        }

                    }).on('error', e => {
                        console.log("unzip error")
                        reject()
                    }).on('finish', e => {
                        allFileCount = currentFileCount
                    })
            rStream.on('error', e=> {
                console.log(e)
                reject(e)
            })
        })
    }

    function uploadFromStream(s3,fileName) {
        var pass = new stream.PassThrough();

        var params = {Bucket: "bucket", Key: "hoge/unzip/" + fileName, Body: pass};
        let request = s3.upload(params, function(err, data) {
            if(err) pass.emit('error')
            if(!err) pass.emit('uploaded')
        })
        request.on('httpUploadProgress', progress => {
            console.log(progress)
        })

        return pass
    }

这是解压缩时使用的库。
https://github.com/mhr3/unzip-stream

This is the library I use when unzipping. https://github.com/mhr3/unzip-stream

帮助我!!

推荐答案

如果你想抓住 NoSuchKey createReadStream抛出的错误你有两个选择:

If you'd like to catch the NoSuchKey error thrown by createReadStream you have 2 options:


  1. 在阅读之前检查密钥是否存在。

  2. 从流中捕获错误

第一个

s3.getObjectMetadata(key)
  .promise()
  .then(() => {
    // This will not throw error anymore
    s3.getObject().createReadStream();
  })
  .catch(error => {
    if (error.statusCode === 404) {
      // Catching NoSuchKey
    }
  });

唯一一种情况,如果文件在瞬间被删除,在解析之间就不会发现错误响应来自 getObjectMetadata 并运行 createReadStream

The only case when you won't catch error if file was deleted in a split second, between parsing response from getObjectMetadata and running createReadStream

第二个

s3.getObject().createReadStream().on('error', error => {
    // Catching NoSuchKey & StreamContentLengthMismatch
});

这是一种更通用的方法,可以捕获所有其他错误,例如网络问题。

This is a more generic approach and will catch all other errors, like network problems.

这篇关于s3.getObject()。createReadStream():如何捕获错误?的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!

查看全文
登录 关闭
扫码关注1秒登录
发送“验证码”获取 | 15天全站免登陆