查询时 Neo4J OutOfMemory 错误 [英] Neo4J OutOfMemory error when querying

问题描述

我最近开始使用 Neo4J，到目前为止我还没有找到我遇到的问题的答案，尤其是服务器方面的问题.我正在使用 1.8.1 版并将服务器作为服务运行在 Windows 上，而不是嵌入的.我的图表有大约 700 万个节点和近 1100 万个关系.

I've recently started working with Neo4J and so far I haven't been able to find an answer to the problems I'm having, in particular with the server. I'm using version 1.8.1 and running the server as a service on Windows, not embedded. The graph I have has around 7m nodes and nearly 11m relationships.

通过小查询和倍数，事情运行得很好.但是，当我尝试撤回更复杂的查询(可能是数千行)时，事情就变糟了.如果我使用控制台，我什么也得不到，然后几分钟或更长时间后出现 undefined 一词(它试图在 Javascript 中执行某些操作，但我不确定是什么).如果我在 .NET 中使用 Neo4JClient，它会超时(我正在通过 WCF 服务处理此问题)并且我怀疑我的问题是服务器端.

With small queries and multiples of, things run nicely. However, when I'm trying to pull back more complex queries, potentially thousands of rows, things go sour. If I'm using the console, I'll get nothing and then after a few minutes or more the word undefined appears (it's trying to do something in Javascript but I'm not sure what). If I'm using Neo4JClient in .NET, it'll time out (I'm working this through a WCF service) and I suspect my problems are server side.

这是一个示例密码查询，它在控制台中引起了我的问题:

Here is a sample cypher query that has caused me problems in the console:

start begin = node:idx(ID="1234")
MATCH begin-[r1?:RELATED_TO]-n1-[r2?:RELATED_TO]-n2-[r3?:RELATED_TO]-n3-[r4?:RELATED_TO]-n4
RETURN begin.Title?, r1.RelationType?, n1.Title?, r2.RelationType?, n2.Title?, r3.RelationType?, n3.Title?, r4.RelationType?, n4.Title?;

我查看了日志，但收到以下严重错误:

I've looked through the logs and I'm receiving the following severe error:

SEVERE: The exception contained within MappableContainerException could not be mapped to a response, re-throwing to the HTTP container
java.lang.OutOfMemoryError: Java heap space
    at java.util.Arrays.copyOf(Unknown Source)
    at java.lang.AbstractStringBuilder.expandCapacity(Unknown Source)
    at java.lang.AbstractStringBuilder.ensureCapacityInternal(Unknown Source)
    at java.lang.AbstractStringBuilder.append(Unknown Source)
    at java.lang.StringBuffer.append(Unknown Source)
    at java.io.StringWriter.write(Unknown Source)
    at java.io.PrintWriter.newLine(Unknown Source)
    at java.io.PrintWriter.println(Unknown Source)
    at java.io.PrintWriter.println(Unknown Source)
    at org.neo4j.cypher.PipeExecutionResult$$anonfun$dumpToString$1.apply(PipeExecutionResult.scala:96)
    at org.neo4j.cypher.PipeExecutionResult$$anonfun$dumpToString$1.apply(PipeExecutionResult.scala:96)
    at scala.collection.LinearSeqOptimized$class.foreach(LinearSeqOptimized.scala:59)
    at scala.collection.immutable.List.foreach(List.scala:45)
    at org.neo4j.cypher.PipeExecutionResult.dumpToString(PipeExecutionResult.scala:96)
    at org.neo4j.cypher.PipeExecutionResult.dumpToString(PipeExecutionResult.scala:124)
    at org.neo4j.cypher.javacompat.ExecutionResult.toString(ExecutionResult.java:90)
    at org.neo4j.shell.kernel.apps.Start.exec(Start.java:72)
    at org.neo4j.shell.kernel.apps.ReadOnlyGraphDatabaseApp.execute(ReadOnlyGraphDatabaseApp.java:32)
    at org.neo4j.shell.impl.AbstractAppServer.interpretLine(AbstractAppServer.java:127)
    at org.neo4j.shell.kernel.GraphDatabaseShellServer.interpretLine(GraphDatabaseShellServer.java:92)
    at org.neo4j.shell.impl.AbstractClient.evaluate(AbstractClient.java:130)
    at org.neo4j.shell.impl.AbstractClient.evaluate(AbstractClient.java:114)
    at org.neo4j.server.webadmin.rest.ShellSession.evaluate(ShellSession.java:96)
    at org.neo4j.server.webadmin.rest.ConsoleService.exec(ConsoleService.java:123)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
    at java.lang.reflect.Method.invoke(Unknown Source)
    at com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)
    at com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$ResponseOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:205)
    at com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75)
    at com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:288)

从有根据的猜测角度看堆栈跟踪，是不是我撤回了太多记录?因为它在扩展 StringBuffer 时内存不足.

From an educated guess perspective looking at the stack trace, is it that I'm pulling back too many records? Since it's running out of memory whilst expanding the StringBuffer.

我想知道 GC 是否可以发挥作用，所以我掌握了 GCViewer.它似乎不是垃圾收集，如果您认为它有用，我可以添加来自 GCViewer 的屏幕截图.

I've wondered whether GC could be playing a part, so I got hold of GCViewer. It doesn't seem to be Garbage Collection, I can add in a screenshot from GCViewer if you think it will be useful.

我已经在默认值和 8G 内存之间的任何地方分配了 JVM.以下是我的配置文件中的一些设置(我将尝试仅包含相关设置).如果您需要更多，请告诉我.

I've allocated the JVM anywhere between the default value and 8G of memory. Here are some of my settings from my configuration files (I'll try and include only the relevant ones). Let me know if you need any more.

Neo4J.properties

Neo4J.properties

# Default values for the low-level graph engine
use_memory_mapped_buffers=false

# Keep logical logs, helps debugging but uses more disk space, enabled for legacy reasons
keep_logical_logs=true

Neo4J-server.properties

Neo4J-server.properties

# HTTP logging is disabled. HTTP logging can be enabled by setting this property to 'true'.
org.neo4j.server.http.log.enabled=false 

Neo4J-Wrapper.conf(可能不熟练地插在一起)

Neo4J-Wrapper.conf (possibly inexpertly slotted together)

# Uncomment the following line to enable garbage collection logging
wrapper.java.additional.4=-Xloggc:data/log/neo4j-gc.log

# Setting a different Garbage Collector as recommended by Neo4J
wrapper.java.additional.5=-XX:+UseConcMarkSweepGC

# other beneficial settings that should boost performance
wrapper.java.additional.6=-d64
wrapper.java.additional.7=-server
wrapper.java.additional.8=-Xss1024k

# Initial Java Heap Size (in MB)
wrapper.java.initmemory=1024

# Maximum Java Heap Size (in MB)
wrapper.java.maxmemory=8000

如有任何帮助，我们将不胜感激.

Any help would be gratefully appreciated.

推荐答案

您的查询太复杂了.当你有这么大的图而不是确保你不会达到你的堆内存限制时，你必须分配适当的内存.你可能想玩一点这个配置: io例子.

your query is simply too complex. when you have such a large graph than to be sure you will not reach your heap memory limit, you must have appropriate memory allocated. you might want to play a little bit with this configuration: io examples.

但是，您的查询可以简化为:

however, your query could be simplified to this:

start begin = node:idx(ID="1234")
MATCH p=begin-[r1:RELATED_TO*0..4]-n4
RETURN p

这篇关于查询时 Neo4J OutOfMemory 错误的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！