如何通过node.js中的JSONStream模块解析以换行符分隔的大JSON文件? [英] how to parse a large, Newline-delimited JSON file by JSONStream module in node.js?

问题描述

我有一个大的json文件，它是用换行符分隔的JSON，其中多个标准JSON对象用多余的换行符分隔，例如

I have a large json file, its is Newline-delimited JSON, where multiple standard JSON objects are delimited by extra newlines, e.g.

{'name':'1','age':5}
{'name':'2','age':3}
{'name':'3','age':6}

我现在在node.js中使用JSONStream解析大型json文件，我之所以使用JSONStream是因为它基于流.

I am now using JSONStream in node.js to parse a large json file, the reason I use JSONStream is because it is based on stream.

但是，示例中的两种语法都无法帮助我用每行中的JSON解析此json文件

However,both parse syntax in the example can't help me to parse this json file with separated JSON in each line

var parser = JSONStream.parse(**['rows', true]**);
var parser = JSONStream.parse([**/./**]);

有人可以帮我吗

推荐答案

警告:由于编写了此答案，因此JSONStream库的作者，显然是为了解决内存泄漏. 将来作为该库的用户，如果需要发出根目录功能，则可以使用0.x.x版本.

Warning: Since this answer was written, the author of the JSONStream library removed the emit root event functionality, apparently to fix a memory leak. Future users of this library, you can use the 0.x.x versions if you need the emit root functionality.

以下是未修改的原始答案:

来自自述文件:

JSONStream.parse(path)

path应该是属性名称，RegExps，布尔值和/或函数的数组.任何与路径匹配的对象都将作为'data'发出.

JSONStream.parse(path)

path should be an array of property names, RegExps, booleans, and/or functions. Any object that matches the path will be emitted as 'data'.

已接收到所有数据时，将发出'root'事件. 'root'事件传递根对象&匹配对象的数量.

A 'root' event is emitted when all data has been received. The 'root' event passes the root object & the count of matched objects.

在您的情况下，由于要获取与特定属性相反的JSON对象，因此将使用'root'事件，而无需指定路径.

In your case, since you want to get back the JSON objects as opposed to specific properties, you will be using the 'root' event and you don't need to specify a path.

您的代码可能看起来像这样:

Your code might look something like this:

var fs = require('fs'),
    JSONStream = require('JSONStream');

var stream = fs.createReadStream('data.json', {encoding: 'utf8'}),
    parser = JSONStream.parse();

stream.pipe(parser);

parser.on('root', function (obj) {
  console.log(obj); // whatever you will do with each JSON object
});

这篇关于如何通过node.js中的JSONStream模块解析以换行符分隔的大JSON文件?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！