写入S3(aws-sdk nodeJS)是否与列出存储桶中的对象冲突? [英] Does writing to S3(aws-sdk nodeJS) conflict with listing objects in a bucket?

问题描述

请记住，充其量我可以说是node和Amazon S3中的新秀.我有一些可以在后台写入S3的应用程序.我想从S3中读取文件已写入的时间，并且只能写入一次.我尝试检查对象数并返回结果:

please bear in mind that I can, at best, be described as a rookie in both node and amazon S3. I have something app that writes to S3 in the background. I want to read from S3 when the file has been written, and only once it's been written. I attempt to check the number of objects and return the result:

function haveFilesBeenWrittenToBucket(bucketName, callback) {
s3.listObjects({ Bucket: bucketName }, function(err, data) {
    const items = data.Contents;
    callback(items);
});

}

和readFile函数:

and the readFile function:

OSClient.prototype.readFile = function(params, callback) {
haveFilesBeenWrittenToBucket(params.Bucket, items => {
    console.log("Number of items " + items.length);
        if (items.length > 0) {
            const rl = readline.createInterface({
                input: s3.getObject(params).createReadStream()
            });
            const myArray = [];
            rl.on("line", function (line) {
                const lineArray = line.split(",");
                for (const value of lineArray) {
                    if (isNaN(value)) {
                        // line.split creates string elements, adding extraneous quotation marks in a string and converting
                        // number to string, so there is a need to reverse this process.
                        const slicedElement = value.slice(1, -1);
                        myArray.push(slicedElement);
                    } else {
                        const valueOfNumber = Number(value);
                        myArray.push(valueOfNumber);
                    }
                }
            })
                .on("close", function () {
                    callback(myArray);

                });
        }
         else{
                var myfunction = this.readFile.bind(this, params, callback);
                setTimeout(myfunction, 5000);
            }


    });

};

最后:

     targetClient.readFile(params, function (arrayResult) {
                            logger.info("Read file:" + fileName + OS_FILE_SUFFIX);
                            readArray = arrayResult;
                        });

如果我在callback(items)上(在'haveFilesBeenWrittenToBucket'中)设置一个断点，则一切正常，并且我取回了存储桶中写入的文件，但如果没有，似乎什么也没写到S3.似乎有些比赛情况，但我真的很笨，我真的很感谢您的帮助.在列出对象和写入S3之间是否存在冲突(至少直到很晚以后，在其他测试中才应该这样做)(这是mocha测试套件的一部分-readFile位于async.waterfall中).我已经经历了好几天了，却一无所获.正如我所说，这是我第一次接触节点，所以请耐心等待我.

If I put a breakpoint on callback(items) (in 'haveFilesBeenWrittenToBucket') everything works fine and I get back the file written in the bucket, but if not, nothing seems to get written to S3. Seems like some race condition, but I'm really clueless and I really would appreciate some help. Is there a conflict between listing objects and writing to S3 (at least not until much later, in some other test, when it shouldn't be (it's part of a mocha test suite - the readFile is in async.waterfall). I have been on this for days and got nowhere. As I said, it's my first exposure to node, so please be patient with me. Thanks.

推荐答案

两件事.首先，事实证明我的问题与nodeJS不相关.叹 其次，API现在提供了一种"waitFor"方法来轮询存储桶或对象是否存在:

Two things. Firstly, it turns out that my issue was not nodeJS related. Sigh Secondly, the API now provides a 'waitFor' method for polling whether a bucket or objects exists:

http://docs.aws.amazon .com/AWSJavaScriptSDK/latest/AWS/S3.html#waitFor-property

这篇关于写入S3(aws-sdk nodeJS)是否与列出存储桶中的对象冲突?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！