Neo4j批量导入“neo4j-admin导入”OutOfMemoryError：Java堆空间和OutOfMemoryError：超出GC开销限制 [英] Neo4j bulk import “neo4j-admin import” OutOfMemoryError: Java heap space and OutOfMemoryError: GC overhead limit exceeded
本文介绍了Neo4j批量导入“neo4j-admin导入”OutOfMemoryError：Java堆空间和OutOfMemoryError：超出GC开销限制的处理方法，对大家解决问题具有一定的参考价值，需要的朋友们下面随着小编来一起学习吧！
问题描述
我的单机可用资源为：
机器总内存：2.00 TB
空闲机器内存：1.81 TB
最大堆内存：910.50 MB
处理器：192
配置的最大内存：1.63 TB
我的file1.csv文件大小为600GB
我的csv文件中的条目数量= 3 000 000 000
头结构
attempt1
item_col1：ID（标签），item_col2，item_col3：IGNORE， item_col4：IGNORE，item_col5，item_col6，item_col7，item_col8：IGNORE
尝试2
item_col1：ID，item_col2，item_col3：IGNORE，item_col4：IGNORE，item_col5，item_col6，item_col7，item_col8：IGNORE
尝试3
item_col1：ID，item_col2，item_col3：IGNORE，item_col4：IGNORE，item_col5：LABEL，item_col6，item_col7，item_col8：IGNORE`
Neo4j版本：3.2.1
试用配置组合1
cat ../conf/neo4j.conf | grepmemory
dbms.memory.heap.initial_size = 16000m
dbms.memory.heap.max_size = 16000m
dbms.memory.pagecache.size = 40g
试过配置组合2
cat ../conf/neo4j.conf | grepmemory
dbms.memory.heap.initial_size = 900m
dbms.memory.heap.max_size = 900m
dbms.memory.pagecache.size = 4g
试过配置组合3
dbms.memory.heap.initial_size = 1000m
dbms.memory.heap.max_size = 1000m
dbms.memory.pagecache.size = 1g
试过配置组合4
dbms.memory.heap。 initial_size = 10g
dbms.memory.heap.max_size = 10g
dbms.memory.pagecache.size = 10g
试过配置组合5（注释）（无输出）
＃dbms.memory.heap .initial_size = 10g
＃dbms.memory.heap.max_size = 10g
＃dbms.memory.pagecache.size = 10g
命令尝试
kaushik @ machine1：/ neo4j / import $ cl
kaushik @ machine1：/ neo4j / import $ rm -r ../data/databases/
kaushik @ machine1：/ neo4j / import $ mkdi r ../data/databases/
kaushik @ machine1：/ neo4j / import $ cat ../conf/neo4j.conf | grep active
dbms.active_database = graph.db
kaushik @ machine1：/ neo4j / import $ ../bin/neo4j-admin import --mode csv - database social.db --nodes head.csv，file1.csv
Neo4j版本：3.2.1
将这些文件的内容导入到/neo4j/data/databases/social.db：
节点：
/neo4j/import/head.csv
/neo4j/import/file1.csv
可用资源：
机器总数内存：2.00 TB
空闲机器内存：1.79 TB
最大堆内存：910.50 MB
处理器：192
配置的最大内存：1.61 TB
错误1
-07-14 05：32：51.736 + 0000
[*节点：7.63 MB ----------------------------- ---------------------- | PROPERTIE | LABEL SCAN --------] 0？ 0
40s内完成439ms
线程main中的异常java.lang.OutOfMemoryError：在org.neo4j.csv.reader.Extractors $ StringArrayExtractor.extract0（提取器）中超出
的GC开销限制。 Java的：739）
在org.neo4j.csv.reader.Extractors $ ArrayExtractor.extract（Extractors.java:680）
在org.neo4j.csv.reader.BufferedCharSeeker.tryExtract（BufferedCharSeeker.java： 239）
at org.neo4j.unsafe.impl.batchimport.input.csv.InputEntityDeserializer.deserializeNextFromSource（InputEntityDeserializer.java:138）
at org.neo4j.unsafe.impl.batchimport.input.csv。 InputEntityDeserializer.fetchNextOrNull（InputEntityDeserializer.java:77）
at org.neo4j.unsafe.impl.batchimport.input.csv.InputEntityDeserializer.fetchNextOrNull（InputEntityDeserializer.java:41）
at org.neo4j.helpers。 collection.PrefetchingIterator.peek（PrefetchingIterator.java:60）
处org.neo4j.unsafe.impl.batchimport org.neo4j.helpers.collection.PrefetchingIterator.hasNext（PrefetchingIterator.java:46）
。 input.csv.Pa rallelInputEntityDeserializer.lambda $ new $ 0（ParallelInputEntityDeserializer.java:106）
at org.neo4j.unsafe.impl.batchimport.input.csv.ParallelInputEntityDeserializer $$ Lambda $ 150 / 1372918763.apply（Unknown Source）
at
线程main中的异常java.lang.OutOfMemoryError：在org.neo4j.csv.reader.Extractors $ StringArrayExtractor.extract0（提取器）中超出
的GC开销限制的.java：739）
在org.neo4j.csv.reader.Extractors $ ArrayExtractor.extract（Extractors.java:680）
在org.neo4j.csv.reader.BufferedCharSeeker.tryExtract（BufferedCharSeeker.java ：239）
at org.neo4j.unsafe.impl.batchimport.input.csv.InputEntityDeserializer.deserializeNextFromSource（InputEntityDeserializer.java:138）
at org.neo4j.unsafe.impl.batchimport.input.csv .InputEntityDeserializer.fetchNextOrNull（InputEntityDeserializer.java:77）
at org.neo4j.unsafe.impl.batchimport.input.csv.InputEntityDeser ializer.fetchNextOrNull（InputEntityDeserializer.java:41）
at org.neo4j.helpers.collection.PrefetchingIterator.peek（PrefetchingIterator.java:60）
at org.neo4j.helpers.collection.PrefetchingIterator.hasNext（ PrefetchingIterator.java:46）
在org.neo4j.unsafe.impl.batchimport.input.csv.ParallelInputEntityDeserializer.lambda美元的新$ 0（ParallelInputEntityDeserializer.java:106）
在org.neo4j.unsafe.impl .batchimport.input.csv.ParallelInputEntityDeserializer $$ LAMBDA $ 150 / 1372918763.apply（未知来源）在org.neo4j.unsafe.impl.batchimport.staging.TicketedProcessing.lambda $
$提交0（TicketedProcessing.java:110）
。在org.neo4j.unsafe.impl.batchimport.staging.TicketedProcessing $$ LAMBDA $ 154 / 1949503798.run（未知来源）
。在org.neo4j.unsafe.impl.batchimport.executor.DynamicTaskExecutor $处理器.run（DynamicTaskExecutor.java:237）
错误3
节点已启动2017-07-14 05：39：48.602 + 0000
[节点：7.63 MB ------------ ----- ------------------------------ | PROPER | * LABEL SCAN -------------- - ] 0？ 0
在42s中完成140ms
线程main中的异常java.lang.OutOfMemoryError：在java.util.Arrays.copyOfRange（Arrays.java:3664）处超出
的GC开销限制
at java.lang.String。< init>（String.java:207）
at org.neo4j.csv.reader.Extractors $ StringExtractor.extract0（Extractors.java:328）
at org.neo4j.csv.reader.Extractors $ AbstractSingleValueExtractor.extract（Extractors.java:287）
在org.neo4j.csv.reader.BufferedCharSeeker.tryExtract（BufferedCharSeeker.java:239）
。在组织。 neo4j.unsafe.impl.batchimport.input.csv.InputEntityDeserializer.deserializeNextFromSource（InputEntityDeserializer.java:138）
at org.neo4j.unsafe.impl.batchimport.input.csv.InputEntityDeserializer.fetchNextOrNull（InputEntityDeserializer.java:77 ）
at org.neo4j.upersafe.impl.batchimport.input.csv.InputEntityDeserializer.fetchNextOrNull（InputEntityDeserializer.java:41）
at org.neo4j.helpers.collection.PrefetchingIterator.peek（PrefetchingIterator.java ：60）
在org.neo4j.helpers.col lection.PrefetchingIterator.hasNext（PrefetchingIterator.java:46）
at org.neo4j.unsafe.impl.batchimport.input.csv.ParallelInputEntityDeserializer.lambda $ new $ 0（ParallelInputEntityDeserializer.java:106）
at org .neo4j.unsafe.impl.batchimport.input.csv.ParallelInputEntityDeserializer $$ LAMBDA $ 150 / 310855317.apply（未知来源）
。在org.neo4j.unsafe.impl.batchimport.staging.TicketedProcessing.lambda $ $提交0（ TicketedProcessing.java:110）
。在org.neo4j.unsafe.impl.batchimport.staging.TicketedProcessing $$ LAMBDA $ 154 / 679112060.run（未知来源）
。在org.neo4j.unsafe.impl.batchimport .executor.DynamicTaskExecutor $ Processor.run（DynamicTaskExecutor.java:237）
错误4
线程main中的异常java.lang.OutOfMemoryError：在org.neo4j.csv.reader.Extractors中超出
的GC开销限制$ StringExtractor.extract0（Extractors.java:328）
org.neo4j.csv.reader.Extractors $ AbstractSingleValueExtractor.extract（Extractors.ja va：287）
at org.neo4j.csv.reader.BufferedCharSeeker.tryExtract（BufferedCharSeeker.java:239）
at org.neo4j.unsafe.impl.batchimport.input.csv.InputEntityDeserializer.deserializeNextFromSource（ InputEntityDeserializer.java:138）
at org.neo4j.unsafe.impl.batchimport.input.csv.InputEntityDeserializer.fetchNextOrNull（InputEntityDeserializer.java:77）
at org.neo4j.unsafe.impl.batchimport。 input.csv.InputEntityDeserializer.fetchNextOrNull（InputEntityDeserializer.java:41）
at org.neo4j.helpers.collection.PrefetchingIterator.peek（PrefetchingIterator.java:60）
at org.neo4j.helpers.collection。 PrefetchingIterator.hasNext（PrefetchingIterator.java:46）
at org.neo4j.unsafe.impl.batchimport.input.csv.ParallelInputEntityDeserializer.lambda $ new $ 0（ParallelInputEntityDeserializer.java:106）
at org.neo4j .unsafe.impl.batchimport.input.csv.ParallelInputEntityDeserializer $$ Lambda $ 118 / 69048864.apply（Unknown Source）
at org.neo4j.unsafe.impl.batchimport.staging.TicketedProcessin g .lambda $ submit $ 0（TicketedProcessing.java:110）
at org.neo4j.unsafe.impl.batchimport.staging.TicketedProcessing $$ Lambda $ 122 / 951451297.run（未知来源）
at org。 neo4j.unsafe.impl.batchimport.executor.DynamicTaskExecutor $ Processor.run（DynamicTaskExecutor.java:237）
Error 5
线程main中的异常java.lang.OutOfMemoryError：Java堆空间$ b $ java.util .Arrays.copyOfRange（Arrays.java:3664）
在java.lang.String。< init>（String.java:207）
at org.neo4j.csv.reader.Extractors $ StringExtractor。 extract0（Extractors.java:328）
在org.neo4j.csv.reader.Extractors $ AbstractSingleValueExtractor.extract（Extractors.java:287）
在org.neo4j.csv.reader.BufferedCharSeeker.tryExtract（ BufferedCharSeeker.java:239）
at org.neo4j.unsafe.impl.batchimport.input.csv.InputEntityDeserializer.deserializeNextFromSource（InputEntityDeserializer.java:138）
at org.neo4j.unsafe.impl.batchimport。 input.csv.InputEntityDese rializer.fetchNextOrNull（InputEntityDeserializer.java:77）
at org.neo4j.unsafe.impl.batchimport.input.csv.InputEntityDeserializer.fetchNextOrNull（InputEntityDeserializer.java:41）
at org.neo4j.helpers。 collection.PrefetchingIterator.peek（PrefetchingIterator.java:60）
处org.neo4j.unsafe.impl.batchimport org.neo4j.helpers.collection.PrefetchingIterator.hasNext（PrefetchingIterator.java:46）
。 input.csv.ParallelInputEntityDeserializer.lambda $ new $ 0（ParallelInputEntityDeserializer.java:106）
at org.neo4j.unsafe.impl.batchimport.input.csv.ParallelInputEntityDeserializer $$ Lambda $ 118 / 950986004.apply（Unknown Source）$在org.neo4j.unsafe.impl.batchimport.staging.TicketedProcessing.lambda b $ b $提交$ 0（TicketedProcessing.java:110）
在org.neo4j.unsafe.impl.batchimport.staging.TicketedProcessing $$ LAMBDA $ 122 / 151277029.run（来源不明） / pre>
一般来说，如果你能解释第9章的性能9.1。以一个例子为例进行内存调优，对很多初学者会有帮助。
https://neo4j.com/docs/operations-manual/current / performance /
你可以举一个例子来计算dbms.memory.heap.initial_size，dbms.memory.heap.max_size，dbms.memory.pagecache .size为500 GB的样本数据集，其中3亿个条目在1TB RAM机器和100个处理器中具有10列相同大小的列。
解决方案实际上，如果您只在做节点，计算非常简单：
3 * 10 ^ 9 * 20/1024 ^ 3
所以我会选择至少55Gb的堆大小。
你可以试试吗？
问候，
汤姆
My single machine available resource is :
Total machine memory: 2.00 TB
Free machine memory: 1.81 TB
Max heap memory : 910.50 MB
Processors: 192
Configured max memory: 1.63 TB
My file1.csv file size is 600GB
Number of entries in my csv file = 3 000 000 000
Header structure
attempt1
item_col1:ID(label),item_col2,item_col3:IGNORE,item_col4:IGNORE,item_col5,item_col6,item_col7,item_col8:IGNORE
Attempt2
item_col1:ID,item_col2,item_col3:IGNORE,item_col4:IGNORE,item_col5,item_col6,item_col7,item_col8:IGNORE
Attempt3
item_col1:ID,item_col2,item_col3:IGNORE,item_col4:IGNORE,item_col5:LABEL,item_col6,item_col7,item_col8:IGNORE`
Neo4j version: 3.2.1
Tried with Configuration combination 1
cat ../conf/neo4j.conf | grep "memory"
dbms.memory.heap.initial_size=16000m
dbms.memory.heap.max_size=16000m
dbms.memory.pagecache.size=40g
Tried with Configuration combination 2
cat ../conf/neo4j.conf | grep "memory"
dbms.memory.heap.initial_size=900m
dbms.memory.heap.max_size=900m
dbms.memory.pagecache.size=4g
Tried with Configuration combination 3
dbms.memory.heap.initial_size=1000m
dbms.memory.heap.max_size=1000m
dbms.memory.pagecache.size=1g
Tried with Configuration combination 4
dbms.memory.heap.initial_size=10g
dbms.memory.heap.max_size=10g
dbms.memory.pagecache.size=10g
Tried with Configuration combination 5 ( commented) (no output)
# dbms.memory.heap.initial_size=10g
# dbms.memory.heap.max_size=10g
# dbms.memory.pagecache.size=10g
Commands tried
kaushik@machine1:/neo4j/import$ cl
kaushik@machine1:/neo4j/import$ rm -r ../data/databases/
kaushik@machine1:/neo4j/import$ mkdir ../data/databases/
kaushik@machine1:/neo4j/import$ cat ../conf/neo4j.conf | grep active
dbms.active_database=graph.db
kaushik@machine1:/neo4j/import$ ../bin/neo4j-admin import --mode csv -- database social.db --nodes head.csv,file1.csv
Neo4j version: 3.2.1
Importing the contents of these files into /neo4j/data/databases/social.db:
Nodes:
/neo4j/import/head.csv
/neo4j/import/file1.csv
Available resources:
Total machine memory: 2.00 TB
Free machine memory: 1.79 TB
Max heap memory : 910.50 MB
Processors: 192
Configured max memory: 1.61 TB
Error 1
Nodes, started 2017-07-14 05:32:51.736+0000
[*NODE:7.63 MB---------------------------------------------------|PROPERTIE|LABEL SCAN--------] 0 ? 0
Done in 40s 439ms
Exception in thread "main" java.lang.OutOfMemoryError: GC overhead limit exceeded
at org.neo4j.csv.reader.Extractors$StringArrayExtractor.extract0(Extractors.java:739)
at org.neo4j.csv.reader.Extractors$ArrayExtractor.extract(Extractors.java:680)
at org.neo4j.csv.reader.BufferedCharSeeker.tryExtract(BufferedCharSeeker.java:239)
at org.neo4j.unsafe.impl.batchimport.input.csv.InputEntityDeserializer.deserializeNextFromSource(InputEntityDeserializer.java:138)
at org.neo4j.unsafe.impl.batchimport.input.csv.InputEntityDeserializer.fetchNextOrNull(InputEntityDeserializer.java:77)
at org.neo4j.unsafe.impl.batchimport.input.csv.InputEntityDeserializer.fetchNextOrNull(InputEntityDeserializer.java:41)
at org.neo4j.helpers.collection.PrefetchingIterator.peek(PrefetchingIterator.java:60)
at org.neo4j.helpers.collection.PrefetchingIterator.hasNext(PrefetchingIterator.java:46)
at org.neo4j.unsafe.impl.batchimport.input.csv.ParallelInputEntityDeserializer.lambda$new$0(ParallelInputEntityDeserializer.java:106)
at org.neo4j.unsafe.impl.batchimport.input.csv.ParallelInputEntityDeserializer$$Lambda$150/1372918763.apply(Unknown Source)
at org.neo4j.unsafe.impl.batchimport
Error 2
Exception in thread "main" java.lang.OutOfMemoryError: GC overhead limit exceeded
at org.neo4j.csv.reader.Extractors$StringArrayExtractor.extract0(Extractors.java:739)
at org.neo4j.csv.reader.Extractors$ArrayExtractor.extract(Extractors.java:680)
at org.neo4j.csv.reader.BufferedCharSeeker.tryExtract(BufferedCharSeeker.java:239)
at org.neo4j.unsafe.impl.batchimport.input.csv.InputEntityDeserializer.deserializeNextFromSource(InputEntityDeserializer.java:138)
at org.neo4j.unsafe.impl.batchimport.input.csv.InputEntityDeserializer.fetchNextOrNull(InputEntityDeserializer.java:77)
at org.neo4j.unsafe.impl.batchimport.input.csv.InputEntityDeserializer.fetchNextOrNull(InputEntityDeserializer.java:41)
at org.neo4j.helpers.collection.PrefetchingIterator.peek(PrefetchingIterator.java:60)
at org.neo4j.helpers.collection.PrefetchingIterator.hasNext(PrefetchingIterator.java:46)
at org.neo4j.unsafe.impl.batchimport.input.csv.ParallelInputEntityDeserializer.lambda$new$0(ParallelInputEntityDeserializer.java:106)
at org.neo4j.unsafe.impl.batchimport.input.csv.ParallelInputEntityDeserializer$$Lambda$150/1372918763.apply(Unknown Source)
at org.neo4j.unsafe.impl.batchimport.staging.TicketedProcessing.lambda$submit$0(TicketedProcessing.java:110)
at org.neo4j.unsafe.impl.batchimport.staging.TicketedProcessing$$Lambda$154/1949503798.run(Unknown Source)
at org.neo4j.unsafe.impl.batchimport.executor.DynamicTaskExecutor$Processor.run(DynamicTaskExecutor.java:237)
Error 3
Nodes, started 2017-07-14 05:39:48.602+0000
[NODE:7.63 MB-----------------------------------------------|PROPER|*LABEL SCAN---------------] 0 ? 0
Done in 42s 140ms
Exception in thread "main" java.lang.OutOfMemoryError: GC overhead limit exceeded
at java.util.Arrays.copyOfRange(Arrays.java:3664)
at java.lang.String.<init>(String.java:207)
at org.neo4j.csv.reader.Extractors$StringExtractor.extract0(Extractors.java:328)
at org.neo4j.csv.reader.Extractors$AbstractSingleValueExtractor.extract(Extractors.java:287)
at org.neo4j.csv.reader.BufferedCharSeeker.tryExtract(BufferedCharSeeker.java:239)
at org.neo4j.unsafe.impl.batchimport.input.csv.InputEntityDeserializer.deserializeNextFromSource(InputEntityDeserializer.java:138)
at org.neo4j.unsafe.impl.batchimport.input.csv.InputEntityDeserializer.fetchNextOrNull(InputEntityDeserializer.java:77)
at org.neo4j.unsafe.impl.batchimport.input.csv.InputEntityDeserializer.fetchNextOrNull(InputEntityDeserializer.java:41)
at org.neo4j.helpers.collection.PrefetchingIterator.peek(PrefetchingIterator.java:60)
at org.neo4j.helpers.collection.PrefetchingIterator.hasNext(PrefetchingIterator.java:46)
at org.neo4j.unsafe.impl.batchimport.input.csv.ParallelInputEntityDeserializer.lambda$new$0(ParallelInputEntityDeserializer.java:106)
at org.neo4j.unsafe.impl.batchimport.input.csv.ParallelInputEntityDeserializer$$Lambda$150/310855317.apply(Unknown Source)
at org.neo4j.unsafe.impl.batchimport.staging.TicketedProcessing.lambda$submit$0(TicketedProcessing.java:110)
at org.neo4j.unsafe.impl.batchimport.staging.TicketedProcessing$$Lambda$154/679112060.run(Unknown Source)
at org.neo4j.unsafe.impl.batchimport.executor.DynamicTaskExecutor$Processor.run(DynamicTaskExecutor.java:237)
Error 4
Exception in thread "main" java.lang.OutOfMemoryError: GC overhead limit exceeded
at org.neo4j.csv.reader.Extractors$StringExtractor.extract0(Extractors.java:328)
at org.neo4j.csv.reader.Extractors$AbstractSingleValueExtractor.extract(Extractors.java:287)
at org.neo4j.csv.reader.BufferedCharSeeker.tryExtract(BufferedCharSeeker.java:239)
at org.neo4j.unsafe.impl.batchimport.input.csv.InputEntityDeserializer.deserializeNextFromSource(InputEntityDeserializer.java:138)
at org.neo4j.unsafe.impl.batchimport.input.csv.InputEntityDeserializer.fetchNextOrNull(InputEntityDeserializer.java:77)
at org.neo4j.unsafe.impl.batchimport.input.csv.InputEntityDeserializer.fetchNextOrNull(InputEntityDeserializer.java:41)
at org.neo4j.helpers.collection.PrefetchingIterator.peek(PrefetchingIterator.java:60)
at org.neo4j.helpers.collection.PrefetchingIterator.hasNext(PrefetchingIterator.java:46)
at org.neo4j.unsafe.impl.batchimport.input.csv.ParallelInputEntityDeserializer.lambda$new$0(ParallelInputEntityDeserializer.java:106)
at org.neo4j.unsafe.impl.batchimport.input.csv.ParallelInputEntityDeserializer$$Lambda$118/69048864.apply(Unknown Source)
at org.neo4j.unsafe.impl.batchimport.staging.TicketedProcessing.lambda$submit$0(TicketedProcessing.java:110)
at org.neo4j.unsafe.impl.batchimport.staging.TicketedProcessing$$Lambda$122/951451297.run(Unknown Source)
at org.neo4j.unsafe.impl.batchimport.executor.DynamicTaskExecutor$Processor.run(DynamicTaskExecutor.java:237)
Error 5
Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
at java.util.Arrays.copyOfRange(Arrays.java:3664)
at java.lang.String.<init>(String.java:207)
at org.neo4j.csv.reader.Extractors$StringExtractor.extract0(Extractors.java:328)
at org.neo4j.csv.reader.Extractors$AbstractSingleValueExtractor.extract(Extractors.java:287)
at org.neo4j.csv.reader.BufferedCharSeeker.tryExtract(BufferedCharSeeker.java:239)
at org.neo4j.unsafe.impl.batchimport.input.csv.InputEntityDeserializer.deserializeNextFromSource(InputEntityDeserializer.java:138)
at org.neo4j.unsafe.impl.batchimport.input.csv.InputEntityDeserializer.fetchNextOrNull(InputEntityDeserializer.java:77)
at org.neo4j.unsafe.impl.batchimport.input.csv.InputEntityDeserializer.fetchNextOrNull(InputEntityDeserializer.java:41)
at org.neo4j.helpers.collection.PrefetchingIterator.peek(PrefetchingIterator.java:60)
at org.neo4j.helpers.collection.PrefetchingIterator.hasNext(PrefetchingIterator.java:46)
at org.neo4j.unsafe.impl.batchimport.input.csv.ParallelInputEntityDeserializer.lambda$new$0(ParallelInputEntityDeserializer.java:106)
at org.neo4j.unsafe.impl.batchimport.input.csv.ParallelInputEntityDeserializer$$Lambda$118/950986004.apply(Unknown Source)
at org.neo4j.unsafe.impl.batchimport.staging.TicketedProcessing.lambda$submit$0(TicketedProcessing.java:110)
at org.neo4j.unsafe.impl.batchimport.staging.TicketedProcessing$$Lambda$122/151277029.run(Unknown Source)
at org.neo4j.unsafe.impl.batchimport.executor.DynamicTaskExecutor$Processor.run(DynamicTaskExecutor.java:237)
In general if you could explain the Chapter 9. Performance 9.1. Memory tuning with an example, it will be helpful for lot of beginners.
https://neo4j.com/docs/operations-manual/current/performance/
could you give an example to calculate dbms.memory.heap.initial_size, dbms.memory.heap.max_size, dbms.memory.pagecache.size for a sample data set of 500 GB with 3Billion entries having 10 columns of equal size in 1TB RAM machine and 100 processors.
解决方案 Actually the calculation is pretty simple if you're only doing nodes :
3 * 10^9 * 20 / 1024^3
So I would go with a heap size of at least 55Gb.
Can you try that ?
Regards,
Tom
这篇关于Neo4j批量导入“neo4j-admin导入”OutOfMemoryError：Java堆空间和OutOfMemoryError：超出GC开销限制的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
查看全文
