如何使用Node.js从Dropbox下载大文件? [英] How to download a big file from Dropbox with Node.js?

问题描述

我想实现大文件下载(大约10-1024 Mb).我已经成功地从Dropbox获取了文件:

I want to implement a big file downloading (approx. 10-1024 Mb). I've already succeeded to get a file from Dropbox:

operationResult = await dbx.filesDownload({
    path: `/${CONFIG_STORAGE.uploader.assetsPath}/${fileUUID}`
});

然后，我将接收到的文件与元数据捆绑在一起，然后将其返回到我的Node.js服务器:

Then I bundle the received file with a meta-data and I return it to my Node.js server:

fileMIME = mime.lookup(operationResult.name);

const downloadResult = Object.freeze({
    fileBinary: operationResult.fileBinary,
    fileLength: operationResult.fileBinary.length,
    fileMIME,
    fileName: operationResult.name,
    isSucceeded,
    message
});

return downloadResult;

现在，我将从Dropbox获得的Buffer转换为Readable流，并将其通过管道返回给客户端:

Now I convert a Buffer, I got from Dropbox, into a Readable stream and pipe it back to a client:

res.setHeader("Content-Disposition", "attachment; filename=" + downloadResult.fileName);
res.setHeader("Content-Type", downloadResult.fileMIME);

const fileReadableStream = new Readable();

fileReadableStream.push(downloadResult.fileBinary);
fileReadableStream.push(null);

fileReadableStream.pipe(res);

到目前为止，所有内容都已清除并且可以正常使用.在这里，我面临第一个陷阱:我需要某种方式来触发浏览器中的下载过程.

Up until now everything is clear and works. Here I face a first pitfall: I need somehow to trigger a download process in browser.

在许多示例中，使用了一些小图片或JSON，我们可以将其完全加载到RAM中，进行操作，例如转换为Base64，将其分配给a.href，然后触发a.click().但是由于我的文件大小为10-50 Mb，所以我不确定这种方法是否正确.

In many examples, some small image or JSON are used, which we can completely load into RAM, make operations, e.g. transforming to Base64, assign it to a.href, and trigger a.click(). But since my file is 10-50 Mb I'm not sure if such approach is a correct one.

我已经尝试过Fetch API:

I've already tried Fetch API:

const response = await fetch(`${host}/download?fileName=${fileName}`, {
    credentials: "same-origin",
    method: "POST",
    mode: "cors"
});

const a = document.createElement("a");
a.href = response.text();
a.download = "MyFile.pdf";
a.click();

但是我总是收到失败-没有文件错误.我还尝试使用jQuery AJAX和XMLHttpRequest(XHR)，但仍然没有文件下载.

But I always get Failed - No file error. I also tried to use jQuery AJAX, and XMLHttpRequest (XHR), but still no file is downloaded.

也许，我缺少一些东西.如何从服务器获取10-1024 Mb文件?

Perhaps, there is something I'm missing. How to get a 10-1024 Mb file from a server?

P.S.我从来没有想到过像文件下载这样的琐碎任务会如此复杂.

P.S. I never thought that such a trivial task, as a file downloading, can be so complicated.

推荐答案

我已通过从filesDownload切换为filesGetTemporaryLink来解决此问题，该解决方案返回指向文件而不是文件本身的链接.然后，我触发该链接的下载.

I've solved the issue by switching from filesDownload to filesGetTemporaryLink, which returns a link to a file instead of the file itself. Then I trigger a downloading of this link.

最终结果:

operationResult = await dbx.filesGetTemporaryLink({
    path: `/${CONFIG_STORAGE.uploader.assetsPath}/${fileUUID}`
});

const downloadResult = Object.freeze({
    fileLength: operationResult?.metadata.size,
    fileLink: operationResult?.link,
    fileMIME: mime.lookup(operationResult?.metadata.name),
    fileName: operationResult?.metadata.name,
    isSucceeded,
    message
});

return downloadResult;

然后我将输出发送给客户端:

Then I send the output to a client:

res.json(downloadResult);

在客户端，我可以通过await/async提取API调用来获取它:

On a client-side I get it via await/async Fetch API call:

const fileResponse = await fetch(``${host}/downloadDocument`, {
    body: JSON.stringify({fileUUID: fileName}),
    cache: "no-cache",
    credentials: "same-origin",
    headers: {
        "Content-Type": "application/json"
    },
    method: "POST",
    mode: "cors"
});

const fileData = await fileResponse.json();

const aTag = document.createElement("a");

aTag.href = fileData.fileLink;
aTag.download = fileData.fileName;
aTag.click();

因此，无论我要下载的文件大小如何，服务器都根本不需要处理文件，也没有额外的CPU，RAM或流量影响.

As a result, a server doesn't have to deal with files at all, no extra CPU, RAM, or traffic impact, no matter what size of a file I'm trying to download.

这篇关于如何使用Node.js从Dropbox下载大文件?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！