NodeJS-请求文件并将其压缩 [英] NodeJS - Request file and zip it

问题描述

我目前正在为我的个人网站创建REST API.我想提供一些下载内容，并希望提供选择多个下载内容并将其作为zip文件下载的可能性.我的第一种方法非常简单:使用网址数组，请求每个网址，将其压缩，发送给用户，删除.但是，考虑到像流之类的东西似乎很适合这个东西，我认为这种方法太脏了.

I am currently in the process of creating a REST API for my personal website. I'd like to include some downloads and I would like to offer the possibility of selecting multiple ones and download those as a zip file. My first approach was pretty easy: Array with urls, request for each of them, zip it, send to user, delete. However, I think that this approach is too dirty considering there are things like streams around which seems to be quite fitting for this thing.

现在，我尝试了一下，目前正在努力处理在不同范围内使用流和事件的基本概念.

Now, I tried around and am currently struggling with the basic concept of working with streams and events throughout different scopes.

以下工作有效:

const r = request(url, options);
r.on('response', function(res) {
    res.pipe(fs.createWriteStream('./file.jpg'));
});

根据我的理解，在这种情况下r是传入的流，我在其上侦听响应事件，一旦发生响应，便将其通过管道传输到用于写入文件系统的流.

From my understanding r is an incoming stream in this scenario and I listen on the response event on it, as soon as it occurs, I pipe it to a stream which I use to write to the file system.

我的第一步是重构它，使其更适合我的情况，但在这里我已经失败了:

My first step was to refactor this so it fits my case more but I already failed here:

async function downloadFile(url) {
    return request({ method: 'GET', uri: url });
}

现在，我想使用一个函数，该函数使用不同的URL调用"downloadFile()"，然后再次使用createWriteStream()将所有这些文件保存到磁盘:

Now I wanted to use a function which calls "downloadFile()" with different urls and save all those files to the disk using createWriteStream() again:

const urls = ['https://download1', 'https://download2', 'https://download3'];
urls.forEach(element => {
    downloadFile(element).then(data => {
        data.pipe(fs.createWriteStream('file.jpg'));
    });
});

使用调试器，我发现数据对象中不存在响应"事件-也许已经是问题了?而且，我发现data.body包含我下载的文档的字节(在这种情况下为pdf)，所以我想知道是否可以将其流式传输到其他地方?

Using the debugger I found out that the "response" event is non existent in the data object -- Maybe that's already the issue? Moreover, I figured that data.body contains the bytes of my downloaded document (a pdf in this case) so I wonder if I could just stream this to some other place?

在阅读了一些stackoveflow线程之后，我找到了以下模块:存档器

After reading some stackoveflow threads I found the following module: archiver

阅读此主题:动态创建zip并将其发送到客户端

@dankohn提出了一种类似的方法:

@dankohn suggested an approach like that:

archive
  .append(fs.createReadStream(file1), { name: 'file1.txt' })
  .append(fs.createReadStream(file2), { name: 'file2.txt' });

让我假设我需要能够从我的数据对象中提取流以继续.

Making me assume I need to be capable of extracting a stream from my data object to proceed.

我在这里走错了路吗?还是我从根本上弄错了什么?

Am I on the wrong track here or am I getting something fundamentally wrong?

lmao感谢您解决我的问题，我不知道发生了什么

lmao thanks for fixing my question I dunno what happened

推荐答案

使用存档器似乎是一种有效的方法，但是，当将大量数据从Web馈送到zip存档时，建议使用流.否则，整个存档数据将需要保存在内存中.

Using archiver seems to be a valid approach, however it would be advisable to use streams when feeding large data from the web into the zip archive. Otherwise, the whole archive data would need to be held in memory.

archiver不支持从流中添加文件，但是 zip-stream 可以.要从网络上读取流，请使用请求.

archiver does not support adding files from streams, but zip-stream does. For reading a stream from the web, request comes in handy.

// npm install -s express zip-stream request

const request = require('request');
const ZipStream = require('zip-stream');
const express = require('express');

const app = express();

app.get('/archive.zip', (req, res) => {
    var zip = new ZipStream()
    zip.pipe(res);

    var stream = request('https://loremflickr.com/640/480')
    zip.entry(stream, { name: 'picture.jpg' }, err => {
        if(err)
            throw err;          
    })

    zip.finalize()
});

app.listen(3000)

更新:使用多个文件的示例

添加一个示例，该示例以递归方式处理 zip.entry()的回调函数中的下一个文件.

Update: Example for using multiple files

Adding an example which processes the next file in the callback function of zip.entry() recursively.

app.get('/archive.zip', (req, res) => {
    var zip = new ZipStream()
    zip.pipe(res);

    var queue = [
        { name: 'one.jpg', url: 'https://loremflickr.com/640/480' },
        { name: 'two.jpg', url: 'https://loremflickr.com/640/480' },
        { name: 'three.jpg', url: 'https://loremflickr.com/640/480' }
    ]

    function addNextFile() {
        var elem = queue.shift()
        var stream = request(elem.url)
        zip.entry(stream, { name: elem.name }, err => {
            if(err)
                throw err;
            if(queue.length > 0)
                addNextFile()
            else
                zip.finalize()
        })
    }

    addNextFile()
})

这篇关于NodeJS-请求文件并将其压缩的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！