node.js:将文本文件读入数组.(每一行都是数组中的一个项目.) [英] node.js: read a text file into an array. (Each line an item in the array.)
问题描述
我想将一个非常非常大的文件读入 node.js 中的 JavaScript 数组.
I would like to read a very, very large file into a JavaScript array in node.js.
所以,如果文件是这样的:
So, if the file is like this:
first line
two
three
...
...
我会有数组:
['first line','two','three', ... , ... ]
函数看起来像这样:
var array = load(filename);
因此将其全部加载为字符串然后将其拆分的想法是不可接受的.
Therefore the idea of loading it all as a string and then splitting it is not acceptable.
推荐答案
如果您可以将最终数据放入一个数组中,那么您是否也可以按照建议的那样将其放入一个字符串中并对其进行拆分?在任何情况下,如果您想一次处理一行文件,您也可以尝试这样的操作:
If you can fit the final data into an array then wouldn't you also be able to fit it in a string and split it, as has been suggested? In any case if you would like to process the file one line at a time you can also try something like this:
var fs = require('fs');
function readLines(input, func) {
var remaining = '';
input.on('data', function(data) {
remaining += data;
var index = remaining.indexOf('
');
while (index > -1) {
var line = remaining.substring(0, index);
remaining = remaining.substring(index + 1);
func(line);
index = remaining.indexOf('
');
}
});
input.on('end', function() {
if (remaining.length > 0) {
func(remaining);
}
});
}
function func(data) {
console.log('Line: ' + data);
}
var input = fs.createReadStream('lines.txt');
readLines(input, func);
(回应 phopkins 的评论)我认为(至少在较新版本中)子字符串不会复制数据,而是创建一个特殊的 SlicedString 对象(来自快速浏览 v8 源代码).在任何情况下,这里都有一个避免提到的子字符串的修改(在一个价值几兆字节的文件上测试所有工作和不玩让杰克成为一个无趣的男孩"):
(in response to comment by phopkins) I think (at least in newer versions) substring does not copy data but creates a special SlicedString object (from a quick glance at the v8 source code). In any case here is a modification that avoids the mentioned substring (tested on a file several megabytes worth of "All work and no play makes Jack a dull boy"):
function readLines(input, func) {
var remaining = '';
input.on('data', function(data) {
remaining += data;
var index = remaining.indexOf('
');
var last = 0;
while (index > -1) {
var line = remaining.substring(last, index);
last = index + 1;
func(line);
index = remaining.indexOf('
', last);
}
remaining = remaining.substring(last);
});
input.on('end', function() {
if (remaining.length > 0) {
func(remaining);
}
});
}
这篇关于node.js:将文本文件读入数组.(每一行都是数组中的一个项目.)的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!