使用node.js保存存储在s3上的图像? [英] Saving an image stored on s3 using node.js?

问题描述

我正在尝试编写一个使用node.js在s3上存储图像的图像服务器.上传图片效果很好，我可以使用s3浏览器客户端正确下载并查看它(特别是我使用的是Dragondisk，但我也已经成功下载了其他图片)，但是当我使用node下载它并尝试要将其写入磁盘，我无法打开文件(文件可能已损坏或使用预览"无法识别的文件格式).我正在使用Amazon sdk作为节点和f​​s来写入文件.我知道您可以将可选的编码传递给fs.writeFile，但是我已经尝试了全部编码，但是它不起作用.我也尝试过在putObject上设置ContentType，在getObject上设置ResponseContentType，以及ContentEncoding和ResponseContentEncoding(以及所有这些东西的各种组合).结果相同.这是一些代码:

I'm trying to write an image server that uses node.js to store images on s3. Uploading the image works fine, and I can download and view it correctly using an s3 browser client (I'm using dragondisk, specifically, but I've successfully downloaded it with other ones too), but when I download it with node and try to write it to disk, I'm unable to open the file (it says it may be damaged or use a file format that Preview does not recognize). I'm using the amazon sdk for node and fs to write the file. I know that you can pass an optional encoding to fs.writeFile, but I've tried them all and it doesn't work. I've also tried setting ContentType on putObject and ResponseContentType on getObject, as well as ContentEncoding and ResponseContentEncoding (and all of these things in various combinations). Same result. Here's some code:

var AWS = require('aws-sdk')
  , gm = require('../lib/gm')
  , uuid = require('node-uui')
  , fs = require('fs');

AWS.config.loadFromPath('./amazonConfig.json');
var s3 = new AWS.S3();

var bucket = 'myBucketName'; // There's other logic here to set the bucket name.

exports.upload = function(req, res) {
    var id = uuid.v4();
    gm.format("/path/to/some/image.jpg", function(format){
        var key = req.params.dir + "/" + id + "/default." + format;
        fs.readFile('/path/to/some/image.jpg', function(err, data){
            if (err) { console.warn(err); }
            else {
                s3.client.putObject({
                    Bucket: bucket,
                    Key: key,
                    Body: data,
                    ContentType: 'image/jpeg'
                    // I've also tried adding ContentEncoding (in various formats) here.
                 }).done(function(response){
                    res.status(200).end(JSON.stringify({ok:1, id: id}));
                }).fail(function(response){
                    res.status(response.httpResponse.statusCode).end(JSON.stringify(({err: response})));
                });
            }
        });
    });
};

exports.get = function(req, res) {
    var key = req.params.dir + "/" + req.params.id + "/default.JPEG";
    s3.client.getObject({
        Bucket: bucket, 
        Key:  key,
        ResponseContentType: 'image/jpeg'
        // Tried ResponseContentEncoding here in base64, binary, and utf8
    }).done(function(response){
        res.status(200).end(JSON.stringify({ok:1, response: response}));
        var filename = '/path/to/new/image/default.JPEG';
        fs.writeFile(filename, response.data.Body, function(err){
            if (err) console.warn(err);
            // This DOES write the file, just not as an image that can be opened.
            // I've tried pretty much every encoding as the optional third parameter
            // and I've matched the encodings to the ResponseContentEncoding and
            // ContentEncoding above (in case it needs to be the same)
        });
    }).fail(function(response){
        res.status(response.httpResponse.statusCode).end(JSON.stringify({err: response}));
    });
};

顺便说一句，我使用express进行路由，所以这就是req.params的来源.

Incidentally, I'm using express for routing, so that's where req.params comes from.

推荐答案

好吧，经过大量的试验和错误之后，我已经弄清楚了如何做到这一点.我最终切换到knox，但是大概可以将类似的策略与aws-sdk结合使用.这种解决方案使我说:必须有一个比这更好的方法."但是，我对任何可行的方法都感到满意.

Ok, after significant trial and error, I've figured out how to do this. I ended up switching to knox, but presumably, you could use a similar strategy with aws-sdk. This is the kind of solution that makes me say, "There has to be a better way than this," but I'm satisfied with anything that works, at this point.

var imgData = "";
client.getFile(key, function(err, fileRes){
    fileRes.on('data', function(chunk){
        imgData += chunk.toString('binary');
    }).on('end', function(){
        res.set('Content-Type', pic.mime);
        res.set('Content-Length', fileRes.headers['content-length']);
        res.send(new Buffer(imgData, 'binary'));
    });
});

getFile()返回数据块作为缓冲区.有人会认为您可以将结果直接传递到前端，但是无论出于何种原因，这都是我可以获得服务以正确返回图像的唯一方法.将缓冲区写入二进制字符串只是将其写回到缓冲区中，这是多余的，但是，嘿，如果可行，它将起作用.如果有人找到了更有效的解决方案，我很想听听.

getFile() returns data chunks as buffers. One would think you could just pipe the results straight to front end, but for whatever reason, this was the ONLY way I could get the service to return an image correctly. It feels redundant to write a buffer to a binary string, only to write it back into a buffer, but hey, if it works, it works. If anyone finds a more efficient solution, I would love to hear it.

这篇关于使用node.js保存存储在s3上的图像?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！