使用Axios的Amazon S3远程文件上传 [英] Amazon S3 Remote File Upload with Axios

问题描述

我正在尝试编写一个函数，该函数将:

I am trying to write a function that would:

1. 将远程URL作为参数，
2. 使用axios获取文件
3. 将流上传到Amazon s3
4. 最后，返回上传的网址

我在stackoverflow上的上找到了帮助.到目前为止，我有这个:

I found help here on stackoverflow. So far, I have this:

/* 
 * Method to pipe the stream 
 */
const uploadFromStream = (file_name, content_type) => {
  const pass = new stream.PassThrough();

  const obj_key = generateObjKey(file_name);
  const params = { Bucket: config.bucket, ACL: config.acl, Key: obj_key, ContentType: content_type, Body: pass };

  s3.upload(params, function(err, data) {
    if(!err){
        return data.Location;
    } else {
        console.log(err, data);
    }
  });

  return pass;
}


/*
 * Method to upload remote file to s3
 */
const uploadRemoteFileToS3 = async (remoteAddr) => {
    axios({
        method: 'get',
        url: remoteAddr,
        responseType: 'stream'
    }).then( (response) => {
        if(response.status===200){
            const file_name = remoteAddr.substring(remoteAddr.lastIndexOf('/')+1);
            const content_type = response.headers['content-type'];
            response.data.pipe(uploadFromStream(file_name, content_type));
        }
    });
}

但是uploadRemoteFileToS3不返回任何内容(因为它是一个异步函数).如何获取上传的网址?

But uploadRemoteFileToS3 does not return anything (because it's a asynchronous function). How can I get the uploaded url?

更新

我进一步改进了代码并编写了一个类.这是我现在所拥有的:

I have further improved upon the code and wrote a class. Here is what I have now:

const config = require('../config.json');

const stream = require('stream');
const axios = require('axios');
const AWS = require('aws-sdk');

class S3RemoteUploader {
    constructor(remoteAddr){
        this.remoteAddr = remoteAddr;
        this.stream = stream;
        this.axios = axios;
        this.config = config;
        this.AWS = AWS;
        this.AWS.config.update({
            accessKeyId: this.config.api_key,
            secretAccessKey: this.config.api_secret
        });
        this.spacesEndpoint = new this.AWS.Endpoint(this.config.endpoint);
        this.s3 = new this.AWS.S3({endpoint: this.spacesEndpoint});

        this.file_name = this.remoteAddr.substring(this.remoteAddr.lastIndexOf('/')+1);
        this.obj_key = this.config.subfolder+'/'+this.file_name;
        this.content_type = 'application/octet-stream';

        this.uploadStream();
    }

    uploadStream(){
        const pass = new this.stream.PassThrough();
        this.promise = this.s3.upload({
            Bucket: this.config.bucket,
            Key: this.obj_key,
            ACL: this.config.acl,
            Body: pass,
            ContentType: this.content_type
        }).promise();
        return pass;
    }

    initiateAxiosCall() {
        axios({
            method: 'get',
            url: this.remoteAddr,
            responseType: 'stream'
        }).then( (response) => {
            if(response.status===200){
                this.content_type = response.headers['content-type'];
                response.data.pipe(this.uploadStream());
            }
        });
    }

    dispatch() {
        this.initiateAxiosCall();
    }

    async finish(){
        //console.log(this.promise); /* return Promise { Pending } */
        return this.promise.then( (r) => {
            console.log(r.Location);
            return r.Location;
        }).catch( (e)=>{
            console.log(e);
        });
    }

    run() {
        this.dispatch();
        this.finish();
    }
}

但是在实现诺言之后，仍然不知道如何获得结果.到目前为止，我尝试了这些:

But still have no clue how to catch the result when the promise is resolved. So far, I tried these:

testUpload = new S3RemoteUploader('https://avatars2.githubusercontent.com/u/41177');
testUpload.run();
//console.log(testUpload.promise); /* Returns Promise { Pending } */
testUpload.promise.then(r => console.log); // does nothing

但以上方法均无效.我有一种感觉，我想念一些非常微妙的东西.有任何线索吗?

But none of the above works. I have a feeling I am missing something very subtle. Any clue, anyone?

推荐答案

上传后，您可以在s3 sdk中调用getsignedurl函数以获取url，还可以在其中指定url的到期时间.您需要传递该功能的密钥.现在旅行将在以后的示例中更新.

After an upload you can call the getsignedurl function in s3 sdk to get the url where you can also specify the expiry of the url as well. You need to pass the key for that function. Now travelling will update with example later.

要生成一个简单的预签名URL，该URL允许任何用户查看 您拥有的存储桶中私有对象的内容，您可以使用 调用getSignedUrl()之后:

To generate a simple pre-signed URL that allows any user to view the contents of a private object in a bucket you own, you can use the following call to getSignedUrl():

 var s3 = new AWS.S3(); 
 var params = {Bucket: 'myBucket', Key: 'myKey'}; 
 s3.getSignedUrl('getObject', params, function (err, url) {  
   console.log("The URL is", url); 
 });

官方文档链接 http://docs.amazonaws.cn/zh_CN/AWSJavaScriptSDK/guide/node-examples.html

代码必须是这样的

function uploadFileToS3AndGenerateUrl(cb) {
const pass = new stream.PassThrough();//I have generated streams from file. Using this since this is what you have used. Must be a valid one.
var params = {
            Bucket: "your-bucket", // required
            Key: key , // required
            Body: pass,
            ContentType: 'your content type',

        };
s3.upload(params, function(s3Err, data) {
    if (s3Err) {
        cb(s3Err)
    }
    console.log(`File uploaded successfully at ${data.Location}`)

    const params = {
        Bucket: 'your-bucket',
        Key: data.key,
        Expires: 180
    };
    s3.getSignedUrl('getObject', params, (urlErr, urlData) => {
        if (urlErr) {

            console.log('There was an error getting your files: ' + urlErr);
            cb(urlErr);

        } else {
            console.log(`url: ${urlData}`);
            cb(null, urlData);

        }
    })
})
}

这篇关于使用Axios的Amazon S3远程文件上传的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！