node.js修改文件数据流? [英] node.js modify file data stream?

问题描述

我需要通过一些修改将一个大数据文件复制到另一个目标. fs.readFile和fs.writeFile非常慢.我需要逐行读取，修改并写入新文件.我发现了这样的东西:

fs.stat(sourceFile, function(err, stat){
    var filesize = stat.size;

    var readStream = fs.createReadStream(sourceFile);

    // HERE I want do some modifications with bytes

    readStream.pipe(fs.createWriteStream(destFile));
})

但是如何进行修改?我试图通过data事件

获取数据

readStream.on('data', function(buffer){
    var str = strToBytes(buffer);
    str.replace('hello', '');
    // How to write ???
});

但是不知道如何将其写入文件:

解决方案

您应使用transform流并使用如下管道:

fs.createReadStream('input/file.txt')
     .pipe(new YourTransformStream())
     .pipe(fs.createWriteStream('output/file.txt'))

然后只需如本文档中那样实现转换流

您还可以像这样使用 scramjet 来简化此操作:

fs.createReadStream('input/file.txt')
     .pipe(new StringStream('utf-8'))
     .split('\n')                                          // split every line
     .map(async (line) => await makeYourChangesTo(line))   // update the lines
     .join('\n')                                           // join again
     .pipe(fs.createWriteStream('output/file.txt'))

我想这比手动操作要容易.

I need to copy one large data file to another destination with some modifications. fs.readFile and fs.writeFile are very slow. I need to read line by line, modify and write to new file. I found something like this:

fs.stat(sourceFile, function(err, stat){
    var filesize = stat.size;

    var readStream = fs.createReadStream(sourceFile);

    // HERE I want do some modifications with bytes

    readStream.pipe(fs.createWriteStream(destFile));
})

But how to make modifications ? I tried to get data with data event

readStream.on('data', function(buffer){
    var str = strToBytes(buffer);
    str.replace('hello', '');
    // How to write ???
});

but don't understand how to write it to file:

解决方案

You should use transform stream and use pipes like this:

fs.createReadStream('input/file.txt')
     .pipe(new YourTransformStream())
     .pipe(fs.createWriteStream('output/file.txt'))

Then it's just a matter of implementing the transform stream as in this doc

You can also make this easier for you using scramjet like this:

fs.createReadStream('input/file.txt')
     .pipe(new StringStream('utf-8'))
     .split('\n')                                          // split every line
     .map(async (line) => await makeYourChangesTo(line))   // update the lines
     .join('\n')                                           // join again
     .pipe(fs.createWriteStream('output/file.txt'))

Which I suppose is easier than doing that manually.

这篇关于node.js修改文件数据流?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！