Node.js fs.writeFile()清空文件 [英] Node.js fs.writeFile() empties the file

问题描述

我有一个更新方法，大约每16-40ms会调用一次，并且在内部有以下代码:

I have an update method which gets called about every 16-40ms, and inside I have this code:

this.fs.writeFile("./data.json", JSON.stringify({
    totalPlayersOnline: this.totalPlayersOnline,
    previousDay: this.previousDay,
    gamesToday: this.gamesToday
}), function (err) {
    if (err) {
        return console.log(err);
    }
});

如果服务器抛出错误，则"data.json"文件有时会变为空.我该如何预防?

If the server throws an error, the "data.json" file sometimes becomes empty. How do I prevent that?

推荐答案

问题

fs.writeFile 不是原子操作.这是一个示例程序，我将在其上运行strace:

#!/usr/bin/env node
const { writeFile, } = require('fs');

// nodejs won’t exit until the Promise completes.
new Promise(function (resolve, reject) {
    writeFile('file.txt', 'content\n', function (err) {
        if (err) {
            reject(err);
        } else {
            resolve();
        }
    });
});

当我在strace -f下运行该命令并整理输出以仅显示writeFile操作的系统调用时(多个IO线程，实际上)，我得到:

When I run that under strace -f and tidied up the output to show just the syscalls from the writeFile operation (which spans multiple IO threads, actually), I get:

open("file.txt", O_WRONLY|O_CREAT|O_TRUNC|O_CLOEXEC, 0666) = 9
pwrite(9, "content\n", 8, 0)            = 8
close(9)                                = 0

如您所见，writeFile分三个步骤完成.

As you can see, writeFile completes in three steps.

1. 该文件已 open() 版本.这是一个原子操作，使用提供的标志在磁盘上创建一个空文件，或者如果该文件存在，则将其截断.截断文件是确保仅写入的内容最终存储在文件中的简便方法.如果文件中已有数据，并且文件比随后写入该文件的数据更长，则多余的数据将保留下来.为避免这种情况，请截断.
2. 内容已写入.因为我写的是这么短的字符串，所以使用单个 pwrite() 调用，但是我认为对于大量数据，nodejs可能一次只能写入一个块.
3. 手柄已关闭.

1. The file is open()ed. This is an atomic operation that, with the provided flags, either creates an empty file on disk or, if the file exists, truncates it. Truncating the file is an easy way to make sure that only the content you write ends up in the file. If there is existing data in the file and the file is longer than the data you subsequently write to the file, the extra data will stay. To avoid this you truncate.
2. The content is written. Because I wrote such a short string, this is done with a single pwrite() call, but for larger amounts of data I assume it is possible nodejs would only write a chunk at a time.
3. The handle is closed.

我的strace的每个步骤都发生在不同的节点IO线程上.这向我暗示fs.writeFile()可能实际上是根据 fs.open() 实施的， fs.write() 和

My strace had each of these steps occurring on a different node IO thread. This suggests to me that fs.writeFile() might actually be implemented in terms of fs.open(), fs.write(), and fs.close(). Thus, nodejs does not treat this complex operation like it is atomic at any level—because it isn’t. Therefore, if your node process terminates, even gracefully, without waiting for the operation to complete, the operation could be at any of the steps above. In your case, you are seeing your process exit after writeFile() finishes step 1 but before it completes step 2.

使用POSIX层以事务方式替换文件内容的常见模式是使用以下步骤:

The common pattern for transactionally replacing a file’s contents with a POSIX layer is to use these steps:

1. 将数据写入一个不同名称的文件 fsync() 文件(请参见确保数据到达磁盘" 中的何时进行fsync?")，然后close()它.
2. rename() (或者，在Windows中， MoveFileEx()与MOVEFILE_REPLACE_EXISTING )与您要替换的文件不同名称的文件.

1. Write the data to a differently named file, fsync() the file (See "When should you fsync?" in "Ensuring data reaches disk"), and then close() it.
2. rename() (or, on Windows, MoveFileEx() with MOVEFILE_REPLACE_EXISTING) the differently-named file over the one you want to replace.

使用此算法，无论程序何时终止，目标文件都会更新或不更新.而且，更好的是，日志记录的(现代)文件系统将确保，只要在继续执行步骤2之前在步骤1中fsync()文件，就可以按顺序进行这两个操作.也就是说，如果您的程序执行了第1步，然后执行了第2步，但您拔掉了插头，那么在启动时，您会发现文件系统处于以下状态之一:

Using this algorithm, the destination file is either updated or not regardless of when your program terminates. And, even better, journalled (modern) filesystems will ensure that, as long as you fsync() the file in step 1 before proceeding to step 2, the two operations will occur in order. I.e., if your program performs step 1 and then step 2 but you pull the plug, when you boot up you will find the filesystem in one of the following states:

• 两个步骤均未完成.原始文件是完整的(或者如果以前从未存在过，则不存在).替换文件不存在(writeFile()算法的步骤1，open()，实际上从未成功)，存在但为空(writeFile()算法的步骤1已完成)，或存在某些数据(算法已部分完成).
• 第一步已完成.原始文件是完整的(或者如果不存在则不存在).替换文件包含所需的所有数据.
• 两个步骤均已完成.现在，在原始文件的路径中，您可以访问替换数据-全部替换数据，而不是空白文件.第一步中写入替换数据的路径已不存在.

• None of the two steps are completed. The original file is intact (or if it never existed before, it doesn’t exist). The replacement file is either nonexistent (step 1 of the writeFile() algorithm, open(), effectively never succeeded), existent but empty (step 1 of writeFile() algorithm completed), or existent with some data (step 2 of writeFile() algorithm partially completed).
• The first step completed. The original file is intact (or if it didn’t exist before it still doesn’t exist). The replacement file exists with all of the data you want.
• Both steps completed. At the path of the original file, you can now access your replacement data—all of it, not a blank file. The path you wrote the replacement data to in the first step no longer exists.

使用此模式的代码可能如下所示:

The code to use this pattern might look like the following:

const { writeFile, rename, } = require('fs');

function writeFileTransactional (path, content, cb) {
    // The replacement file must be in the same directory as the
    // destination because rename() does not work across device
    // boundaries.

    // This simple choice of replacement filename means that this
    // function must never be called concurrently with itself for the
    // same path value. Also, properly guarding against other
    // processes trying to use the same temporary path would make this
    // function more complicated. If that is a concern, a proper
    // temporary file strategy should be used. However, this
    // implementation ensures that any files left behind during an 
    //unclean termination will be cleaned up on a future run.
    let temporaryPath = `${path}.new`;
    writeFile(temporaryPath, content, function (err) {
        if (err) {
            return cb(err);
        }

        rename(temporaryPath, path, cb);
    });
};

这基本上是您在任何语言/框架中用于解决相同问题的相同解决方案.

This is basically the same solution you’d use for the same problem in any langage/framework.

这篇关于Node.js fs.writeFile()清空文件的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！