如何使用适用于 Node.js 的 AWS 开发工具包将 Amazon S3 中的所有对象从一个前缀复制/移动到另一个前缀 [英] How to copy/move all objects in Amazon S3 from one prefix to other using the AWS SDK for Node.js

问题描述

如何将所有对象从一个前缀复制到另一个前缀?我已经尝试了所有可能的方法来将一个镜头中的所有对象从一个前缀复制到另一个，但似乎唯一可行的方法是循环遍历对象列表并一个一个地复制它们.这真的是低效的.如果我在一个文件夹中有数百个文件，我是否需要进行 100 次调用?

How do I copy all objects from one prefix to other? I have tried all possible ways to copy all objects in one shot from one prefix to other, but the only way that seems to work is by looping over a list of objects and copying them one by one. This is really inefficient. If I have hundreds of files in a folder, will I have to make 100 calls?

var params = {
         Bucket: bucket,
         CopySource: bucket+'/'+oldDirName+'/filename.txt',
         Key: newDirName+'/filename.txt',
 };
s3.copyObject(params, function(err, data) {
  if (err) {
      callback.apply(this, [{
          type: "error",
          message: "Error while renaming Directory",
          data: err
      }]);
  } else {
      callback.apply(this, [{
          type: "success",
          message: "Directory renamed successfully",
          data: data
      }]);
  }
});

推荐答案

您将需要制作一个 AWS.S3.listObjects() 以列出具有特定前缀的对象.但是您是正确的，因为您需要为要从一个存储桶/前缀复制到相同或另一个存储桶/前缀的每个对象进行一次调用.

You will need to make one AWS.S3.listObjects() to list your objects with a specific prefix. But you are correct in that you will need to make one call for every object that you want to copy from one bucket/prefix to the same or another bucket/prefix.

您还可以使用诸如 async 之类的实用程序库来管理您的请求.

You can also use a utility library like async to manage your requests.

var AWS = require('aws-sdk');
var async = require('async');
var bucketName = 'foo';
var oldPrefix = 'abc/';
var newPrefix = 'xyz/';
var s3 = new AWS.S3({params: {Bucket: bucketName}, region: 'us-west-2'});

var done = function(err, data) {
  if (err) console.log(err);
  else console.log(data);
};

s3.listObjects({Prefix: oldPrefix}, function(err, data) {
  if (data.Contents.length) {
    async.each(data.Contents, function(file, cb) {
      var params = {
        Bucket: bucketName,
        CopySource: bucketName + '/' + file.Key,
        Key: file.Key.replace(oldPrefix, newPrefix)
      };
      s3.copyObject(params, function(copyErr, copyData){
        if (copyErr) {
          console.log(copyErr);
        }
        else {
          console.log('Copied: ', params.Key);
          cb();
        }
      });
    }, done);
  }
});

希望这有帮助！

这篇关于如何使用适用于 Node.js 的 AWS 开发工具包将 Amazon S3 中的所有对象从一个前缀复制/移动到另一个前缀的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！