JSON.stringify大对象优化 [英] JSON.stringify large object optimization

问题描述

我想使用 JSON.stringify 将大型JSON对象转换为字符串，但是由于对象的大小，我得到了

I'd like to convert a large JSON object to string using the JSON.stringify, but due to the size of the object I got an error of

<--- Last few GCs --->

[20817:0x2cc2830]   295727 ms: Scavenge 1335.8 (1423.9) -> 1335.6 (1427.9) MB, 7.7 / 0.0 ms  (average mu = 0.255, current mu = 0.170) allocation failure 
[20817:0x2cc2830]   295966 ms: Mark-sweep 1339.5 (1427.9) -> 1339.3 (1422.9) MB, 227.1 / 0.0 ms  (average mu = 0.272, current mu = 0.291) allocation failure scavenge might not succeed
[20817:0x2cc2830]   295990 ms: Scavenge 1343.2 (1422.9) -> 1343.0 (1426.9) MB, 9.2 / 0.0 ms  (average mu = 0.272, current mu = 0.291) allocation failure 


<--- JS stacktrace --->

==== JS stack trace =========================================

Security context: 0x16f9b389e549 <JSObject>
    0: builtin exit frame: stringify(this=0x16f9b38877f9 <Object map = 0x39fb09082ba1>,0x1da31d6826f1 <undefined>,0x2762ba9be041 <JSFunction replacer (sfi = 0x1bc744b67dc9)>,0x1deddcf2e701 <Object map = 0x6bcf8da0341>,0x16f9b38877f9 <Object map = 0x39fb09082ba1>)

    1: stringify [0x16c685254241] [/home/blingga/Documents/integrations/integration-flow17employment-hrplus-to-datalake/dist/flow17em...

FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory
 1: 0x8c02c0 node::Abort() [node]
 2: 0x8c030c  [node]
 3: 0xad15de v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, bool) [node]
 4: 0xad1814 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [node]
 5: 0xebe752  [node]
 6: 0xebe858 v8::internal::Heap::CheckIneffectiveMarkCompact(unsigned long, double) [node]
 7: 0xeca982 v8::internal::Heap::PerformGarbageCollection(v8::internal::GarbageCollector, v8::GCCallbackFlags) [node]
 8: 0xecb2b4 v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [node]
 9: 0xecdf21 v8::internal::Heap::AllocateRawWithRetryOrFail(int, v8::internal::AllocationSpace, v8::internal::AllocationAlignment) [node]
10: 0xe962a5  [node]
11: 0xe9dc0b v8::internal::Factory::NewRawTwoByteString(int, v8::internal::PretenureFlag) [node]
12: 0x11cd6b0 v8::internal::IncrementalStringBuilder::Extend() [node]
13: 0xf92c20 v8::internal::JsonStringifier::Result v8::internal::JsonStringifier::Serialize_<true>(v8::internal::Handle<v8::internal::Object>, bool, v8::internal::Handle<v8::internal::Object>) [node]
14: 0xf9719d v8::internal::JsonStringifier::Result v8::internal::JsonStringifier::Serialize_<false>(v8::internal::Handle<v8::internal::Object>, bool, v8::internal::Handle<v8::internal::Object>) [node]
15: 0xf977ee v8::internal::JsonStringifier::SerializeArrayLikeSlow(v8::internal::Handle<v8::internal::JSReceiver>, unsigned int, unsigned int) [node]
16: 0xf93e96 v8::internal::JsonStringifier::Result v8::internal::JsonStringifier::Serialize_<true>(v8::internal::Handle<v8::internal::Object>, bool, v8::internal::Handle<v8::internal::Object>) [node]
17: 0xf9719d v8::internal::JsonStringifier::Result v8::internal::JsonStringifier::Serialize_<false>(v8::internal::Handle<v8::internal::Object>, bool, v8::internal::Handle<v8::internal::Object>) [node]
18: 0xf977ee v8::internal::JsonStringifier::SerializeArrayLikeSlow(v8::internal::Handle<v8::internal::JSReceiver>, unsigned int, unsigned int) [node]
19: 0xf93e96 v8::internal::JsonStringifier::Result v8::internal::JsonStringifier::Serialize_<true>(v8::internal::Handle<v8::internal::Object>, bool, v8::internal::Handle<v8::internal::Object>) [node]
20: 0xf9719d v8::internal::JsonStringifier::Result v8::internal::JsonStringifier::Serialize_<false>(v8::internal::Handle<v8::internal::Object>, bool, v8::internal::Handle<v8::internal::Object>) [node]
21: 0xf97ef9 v8::internal::JsonStringifier::Stringify(v8::internal::Handle<v8::internal::Object>, v8::internal::Handle<v8::internal::Object>, v8::internal::Handle<v8::internal::Object>) [node]
22: 0xba0051 v8::internal::Builtin_JsonStringify(int, v8::internal::Object**, v8::internal::Isolate*) [node]
23: 0x2639339dc17d 
Aborted (core dumped)

有什么方法可以优化 JSON.stringify ?还是无论如何我都可以将JSON转换为字符串?谢谢.

Is there any way I can optimize the JSON.stringify? Or is there anyway I can convert my JSON to string? Thanks.

推荐答案

您尝试 big-json ?

const fs = require('fs');
const path = require('path');
const json = require('big-json');

const readStream = fs.createReadStream('big.json');
const parseStream = json.createParseStream();

parseStream.on('data', function(pojo) {
    // => receive reconstructed POJO
});

readStream.pipe(parseStream);

这篇关于JSON.stringify大对象优化的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！