为什么fs.readFileSync()比等待fsPromises.readFile()更快? [英] Why is fs.readFileSync() faster than await fsPromises.readFile()?

问题描述

这是测试代码(在express环境中，仅仅是因为这就是我碰到的东西):

Here is the test code (in an express environment just because that's what I happen to be messing around with):

const fs = require('fs-extra');
const fsPromises = fs.promises;
const express = require('express');
const app = express();

const speedtest = async function (req, res, next) {
    const useFsPromises = (req.params.promises == 'true');
    const jsonFileName = './json/big-file.json';
    const hrstart = process.hrtime();
    if (useFsPromises) {
        await fsPromises.readFile(jsonFileName);
    } else {
        fs.readFileSync(jsonFileName);
    }
    res.send(`time taken to read: ${process.hrtime(hrstart)[1]/1000000} ms`);
};

app.get('/speedtest/:promises', speedtest);

big-file.json文件约为16 MB.使用节点12.18.4.

The big-file.json file is around 16 MB. Using node 12.18.4.

典型结果(这些值之间有很大差异，但以下是典型"):

Typical results (varies quite a bit around these values, but the following are "typical"):

https://dev.mydomain.com/speedtest/false
time taken to read: 3.948152 ms

https://dev.mydomain.com/speedtest/true
time taken to read: 61.865763 ms

更新，以包括另外两个变体...普通的fs.readFile()以及该版本的约定版本:

UPDATE to include two more variants... plain fs.readFile() and also a promisified version of this:

const fs = require('fs-extra');
const fsPromises = fs.promises;
const util = require('util');
const readFile = util.promisify(fs.readFile);
const express = require('express');
const app = express();

const speedtest = async function (req, res, next) {
    const type = req.params.type;
    const jsonFileName = './json/big-file.json';
    const hrstart = process.hrtime();
    if (type == 'readFileFsPromises') {
        await fsPromises.readFile(jsonFileName);
    } else if (type == 'readFileSync') {
        fs.readFileSync(jsonFileName);
    } else if (type == 'readFileAsync') {
        return fs.readFile(jsonFileName, function (err, jsondata) {
            res.send(`time taken to read: ${process.hrtime(hrstart)[1]/1000000} ms`);
        });
    } else if (type == 'readFilePromisified') {
        await readFile(jsonFileName);
    }
    res.send(`time taken to read: ${process.hrtime(hrstart)[1]/1000000} ms`);
};

app.get('/speedtest/:type', speedtest);

我发现fsPromises.readFile()是最慢的，而其他的则要快得多，并且在阅读时间方面都大致相同.我应该在不同的示例中添加它(我无法完全验证，所以我不确定发生了什么)时差比此处报告的要大得多.目前在我看来，应该简单地避免使用fsPromises.readFile()，因为还有其他异步/承诺选项.

I am finding that the fsPromises.readFile() is the slowest, while the others are much faster and all roughly the same in terms of reading time. I should add that in a different example (which I can't fully verify so I'm not sure what was going on) the time difference was vastly bigger than reported here. Seems to me at present that fsPromises.readFile() should simply be avoided because there are other async/promise options.

推荐答案

在调试器(fs.readFileSync和fs.promises.readFile)中逐步完成每个实现之后，我可以确认同步版本会大块读取整个文件. (文件大小).而fs.promises.readFile()一次循环读取16384个字节，每次读取await.这将使fs.promises.readFile()多次返回事件循环，然后才能读取整个文件.除了让其他事物有运行的机会外，还需要额外的开销才能通过for循环在每个循环中返回事件循环.还有内存管理开销，因为fs.promises.readFile()分配了一系列Buffer对象，然后在结尾处将它们全部组合在一起，而fs.readFileSync()在开头分配了一个大Buffer对象，然后将整个文件读入该Buffer中.

After stepping through each implementation in the debugger (fs.readFileSync and fs.promises.readFile), I can confirm that the synchronous version reads the entire file in one large chunk (the size of the file). Whereas fs.promises.readFile() reads 16,384 bytes at a time in a loop, with an await on each read. This is going to make fs.promises.readFile() go back to the event loop multiple times before it can read the entire file. Besides giving other things a chance to run, it's extra overhead to go back to the event loop every cycle through a for loop. There's also memory management overhead because fs.promises.readFile() allocates a series of Buffer objects and then combines them all at the end, whereas fs.readFileSync() allocates one large Buffer object at the beginning and just reads the entire file into that one Buffer.

因此，从纯时间到完成的角度来看，允许占用整个CPU的同步版本(从多用户服务器中使用的CPU周期来看，效率明显较低)，因为它会阻止事件循环在读取过程中执行其他任何操作).异步版本正在读取较小的块，可能是为了避免过多地阻塞事件循环，以便其他内容可以在fs.promises.readFile()执行其操作时有效地交织并运行.

So, the synchronous version, which is allowed to hog the entire CPU, is just faster from a pure time to completion point of view (it's significantly less efficient from a CPU cycles used point of view in a multi-user server because it blocks the event loop from doing anything else during the read). The asynchronous version is reading in smaller chunks, probably to avoid blocking the event loop too much so other things can effectively interleave and run while fs.promises.readFile() is doing its thing.

对于我前一段时间从事的项目，我编写了自己的readFile()简单异步版本，该版本一次读取整个文件，并且比内置实现快得多.我不关心那个特定项目中的事件循环阻塞，因此我没有调查这是否是一个问题.

For a project I worked on awhile ago, I wrote my own simple asynchronous version of readFile() that reads the entire file at once and it was significantly faster than the built-in implementation. I was not concerned about event loop blockage in that particular project so I did not investigate if that's an issue.

此外，fs.readFile()仅使用简单的回调以524,288字节的块(比fs.promises.readFile()大得多的块)读取文件，并且不使用await.显然，它的编码比promise实现的编码更优化.我不知道为什么他们为fs.promises.readFile()实现以较慢的方式重写了该功能.目前看来，用承诺包装fs.readFile()会更快.

In addition, fs.readFile() reads the file in 524,288 byte chunks (much larger chunks that fs.promises.readFile()) and does not use await, using just plain callbacks. It is apparently just coded more optimally than the promise implementation. I don't know why they rewrote the function in a slower way for the fs.promises.readFile() implementation. For now, it appears that wrapping fs.readFile() with a promise would be faster.

这篇关于为什么fs.readFileSync()比等待fsPromises.readFile()更快?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！