查询时出现Neo4J OutOfMemory错误 [英] Neo4J OutOfMemory error when querying

问题描述

我最近开始使用Neo4J，到目前为止，我还无法找到我所遇到的问题的答案，尤其是服务器方面的问题.我正在使用1.8.1版，并在Windows(而非嵌入式)上将服务器作为服务运行.我拥有的图大约有7m个节点，并且有近11m个关系.

I've recently started working with Neo4J and so far I haven't been able to find an answer to the problems I'm having, in particular with the server. I'm using version 1.8.1 and running the server as a service on Windows, not embedded. The graph I have has around 7m nodes and nearly 11m relationships.

通过小的查询和多个查询，事情就可以很好地运行.但是，当我尝试撤回更复杂的查询(可能有数千行)时，事情就变糟了.如果使用控制台，则什么都不会得到，然后在几分钟或更长时间后出现undefined字样(它正在尝试用Javascript做某事，但我不确定该怎么做).如果我在.NET中使用Neo4JClient，它将超时(我正在通过WCF服务进行此操作)，并且我怀疑我的问题是在服务器端.

With small queries and multiples of, things run nicely. However, when I'm trying to pull back more complex queries, potentially thousands of rows, things go sour. If I'm using the console, I'll get nothing and then after a few minutes or more the word undefined appears (it's trying to do something in Javascript but I'm not sure what). If I'm using Neo4JClient in .NET, it'll time out (I'm working this through a WCF service) and I suspect my problems are server side.

这是一个示例密码查询，已导致我在控制台中遇到问题:

Here is a sample cypher query that has caused me problems in the console:

start begin = node:idx(ID="1234")
MATCH begin-[r1?:RELATED_TO]-n1-[r2?:RELATED_TO]-n2-[r3?:RELATED_TO]-n3-[r4?:RELATED_TO]-n4
RETURN begin.Title?, r1.RelationType?, n1.Title?, r2.RelationType?, n2.Title?, r3.RelationType?, n3.Title?, r4.RelationType?, n4.Title?;

我已经查看了日志，并收到以下严重错误:

I've looked through the logs and I'm receiving the following severe error:

SEVERE: The exception contained within MappableContainerException could not be mapped to a response, re-throwing to the HTTP container
java.lang.OutOfMemoryError: Java heap space
    at java.util.Arrays.copyOf(Unknown Source)
    at java.lang.AbstractStringBuilder.expandCapacity(Unknown Source)
    at java.lang.AbstractStringBuilder.ensureCapacityInternal(Unknown Source)
    at java.lang.AbstractStringBuilder.append(Unknown Source)
    at java.lang.StringBuffer.append(Unknown Source)
    at java.io.StringWriter.write(Unknown Source)
    at java.io.PrintWriter.newLine(Unknown Source)
    at java.io.PrintWriter.println(Unknown Source)
    at java.io.PrintWriter.println(Unknown Source)
    at org.neo4j.cypher.PipeExecutionResult$$anonfun$dumpToString$1.apply(PipeExecutionResult.scala:96)
    at org.neo4j.cypher.PipeExecutionResult$$anonfun$dumpToString$1.apply(PipeExecutionResult.scala:96)
    at scala.collection.LinearSeqOptimized$class.foreach(LinearSeqOptimized.scala:59)
    at scala.collection.immutable.List.foreach(List.scala:45)
    at org.neo4j.cypher.PipeExecutionResult.dumpToString(PipeExecutionResult.scala:96)
    at org.neo4j.cypher.PipeExecutionResult.dumpToString(PipeExecutionResult.scala:124)
    at org.neo4j.cypher.javacompat.ExecutionResult.toString(ExecutionResult.java:90)
    at org.neo4j.shell.kernel.apps.Start.exec(Start.java:72)
    at org.neo4j.shell.kernel.apps.ReadOnlyGraphDatabaseApp.execute(ReadOnlyGraphDatabaseApp.java:32)
    at org.neo4j.shell.impl.AbstractAppServer.interpretLine(AbstractAppServer.java:127)
    at org.neo4j.shell.kernel.GraphDatabaseShellServer.interpretLine(GraphDatabaseShellServer.java:92)
    at org.neo4j.shell.impl.AbstractClient.evaluate(AbstractClient.java:130)
    at org.neo4j.shell.impl.AbstractClient.evaluate(AbstractClient.java:114)
    at org.neo4j.server.webadmin.rest.ShellSession.evaluate(ShellSession.java:96)
    at org.neo4j.server.webadmin.rest.ConsoleService.exec(ConsoleService.java:123)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
    at java.lang.reflect.Method.invoke(Unknown Source)
    at com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)
    at com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$ResponseOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:205)
    at com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75)
    at com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:288)

从有根据的猜测角度看堆栈跟踪，是否是我要拉回太多记录?由于它在扩展StringBuffer时内存不足.

From an educated guess perspective looking at the stack trace, is it that I'm pulling back too many records? Since it's running out of memory whilst expanding the StringBuffer.

我想知道GC是否可以发挥作用，所以我掌握了GCViewer.它似乎不是垃圾收集，如果您认为有用的话，我可以在GCViewer中添加一个屏幕截图.

I've wondered whether GC could be playing a part, so I got hold of GCViewer. It doesn't seem to be Garbage Collection, I can add in a screenshot from GCViewer if you think it will be useful.

我已经在默认值和8G内存之间的任何位置分配了JVM.这是我的配置文件中的一些设置(我将尝试并仅包括相关的设置).让我知道是否需要更多的东西.

I've allocated the JVM anywhere between the default value and 8G of memory. Here are some of my settings from my configuration files (I'll try and include only the relevant ones). Let me know if you need any more.

Neo4J.properties

Neo4J.properties

# Default values for the low-level graph engine
use_memory_mapped_buffers=false

# Keep logical logs, helps debugging but uses more disk space, enabled for legacy reasons
keep_logical_logs=true

Neo4J-server.properties

Neo4J-server.properties

# HTTP logging is disabled. HTTP logging can be enabled by setting this property to 'true'.
org.neo4j.server.http.log.enabled=false 

Neo4J-Wrapper.conf(可能不是很专业地放在一起)

Neo4J-Wrapper.conf (possibly inexpertly slotted together)

# Uncomment the following line to enable garbage collection logging
wrapper.java.additional.4=-Xloggc:data/log/neo4j-gc.log

# Setting a different Garbage Collector as recommended by Neo4J
wrapper.java.additional.5=-XX:+UseConcMarkSweepGC

# other beneficial settings that should boost performance
wrapper.java.additional.6=-d64
wrapper.java.additional.7=-server
wrapper.java.additional.8=-Xss1024k

# Initial Java Heap Size (in MB)
wrapper.java.initmemory=1024

# Maximum Java Heap Size (in MB)
wrapper.java.maxmemory=8000

我们将不胜感激任何帮助.

Any help would be gratefully appreciated.

推荐答案

您的查询太简单了.当您有一个很大的图形而不是确保您不会达到堆内存限制时，必须分配适当的内存.您可能需要使用以下配置: io例子.

your query is simply too complex. when you have such a large graph than to be sure you will not reach your heap memory limit, you must have appropriate memory allocated. you might want to play a little bit with this configuration: io examples.

但是，您的查询可以简化为:

however, your query could be simplified to this:

start begin = node:idx(ID="1234")
MATCH p=begin-[r1:RELATED_TO*0..4]-n4
RETURN p

这篇关于查询时出现Neo4J OutOfMemory错误的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！