Neo4j节点创建速度 [英] Neo4j node creation speed

问题描述

我在笔记本电脑上进行了新的neo4j设置，并且通过REST API创建新节点的速度似乎很慢(平均约30-40毫秒).我已经在Google上搜索了一下，但是找不到应该"花费多长时间的真正基准. 这篇文章，但这仅列出相对性能，而不是绝对性能. neo4j固有地限于每秒仅添加〜30个新节点(批处理模式之外)，还是我的配置有问题?

I have a fresh neo4j setup on my laptop, and creating new nodes via the REST API seems to be quite slow (~30-40 ms average). I've Googled around a bit, but can't find any real benchmarks for how long it "should" take; there's this post, but that only lists relative performance, not absolute performance. Is neo4j inherently limited to only adding ~30 new nodes per second (outside of batch mode), or is there something wrong with my configuration?

配置详细信息:

• Neo4j版本2.2.5
• 服务器在我的2014年中期笔记本电脑上，运行Ubuntu 15.04
• OpenJDK 1.8版
• 服务器的呼叫也来自我的笔记本电脑(通过localhost:7474)，因此不应涉及任何网络延迟
• 我正在通过Clojure/Neocons呼叫neo4j；使用的方法是clojurewerkz.neocons.rest.nodes类中的创建"
• 使用Cypher似乎更慢；例如.通过HTML界面调用"PROFILE CREATE(you:Person {name:"Jane Doe"}})RETURN YOU会返回"Cypher版本:CYPHER 2.2，计划程序:RULE.在54毫秒内5次总数据库匹配."

推荐答案

Neo4j的性能特点是一个棘手的领域.

Neo4j performance charasteristics is a tricky area.

首先:这完全取决于服务器的配置方式.在笔记本电脑上测量任何东西都是错误的方法.

First of all: it all depends a lot on how server is configured. Measuring anything on laptop is wrong way to do it.

在评估性能之前，应检查以下内容:

Befor measuring performance you should check following:

1. 您具有适当的服务器硬件(要求)
2. 客户端和服务器在本地网络中.
3. Neo4j的配置正确(内存映射，Web服务器线程池，Java堆大小等)
4. 服务器已正确配置(Linux tcp堆栈，最大可用打开文件等)
5. 服务器已预热. Neo4j是用Java编写的，因此您应该在测量数字之前进行适当的预热(即，进行约15分钟的加载).

1. You have appropriate server hardware (requirements)
2. Client and server are in local network.
3. Neo4j is properly configured (memory mapping, webserver thread pool, java heap size and etc)
4. Server is properly configured (Linux tcp stack, maximum available open files and etc)
5. Server is warmed up. Neo4j is written in Java, so you should do appropriate warmup before measuring numbers (i.e. making some load for ~15 minutes).

最后一个-企业版. Neo4j企业版具有一些可以大大提高性能的高级功能(即 HPC缓存 ).

And last one - enterprise edition. Neo4j enterprise edition has some advanced features that can improve performance a lot (i.e. HPC cache).

Neo4j内部为:

• 存储
• 核心API
• Traversal API
• 密码API

无需执行任何其他网络请求即可执行所有操作. Neo4j 服务器是建立在这个坚实基础之上的.

Everything is performed without any additional network requests. Neo4j server is build on top of this solid foundation.

因此，当您向Neo4j服务器发出请求时，您正在测量:

So, when you are making request to Neo4j server, you are measuring:

• 客户端和服务器之间的延迟
• JSON序列化费用
• Web服务器(码头)
• 用于管理锁，事务等的其他模块
• 还有Neo4j本身

因此，这里的底线是-如果在嵌入式模式下使用，Neo4j本身就非常快.但是与Neo4j服务器打交道会涉及额外的费用.

So, bottom line here is - Neo4j is pretty fast by itself, if used in embedded mode. But dealing with Neo4j server involved additional costs.

我们进行了内部Neo4j测试.我们测量了几个案例.

We had internal Neo4j testing. We measured several cases.

在这里，我们使用的是Vanilla Transactional Cypher REST API.

Here we are using vanilla Transactional Cypher REST API.

线程:2

Node per transaction: 1000  
Execution time: 1635  
Total nodes created: 7000000  
Nodes per second: 7070  

线程:5

Node per transaction: 750  
Execution time: 852  
Total nodes created: 7000000  
Nodes per second: 8215  

巨大的数据库同步

此版本使用定制开发的非托管扩展，并且之间使用二进制协议服务器和客户端以及一些并发性.

Huge database sync

This one uses custom developed unmanaged extension, with binary protocol between server and client and some concurrency.

但这仍然是Neo4j服务器(实际上是Neo4j群集).

But this is still Neo4j server (in fact - Neo4j cluster).

Node count: 80.32M (80 320 000)
Relationship count: 80.30M (80 300 000)
Property count: 257.78M (257 780 000)
Consumed time: 2142 seconds

Per second:
Nodes - 37497
Relationships - 37488
Properties - 120345

此数字显示了Neo4j的真正威力.

This numbers shows true Neo4j power.

我现在尝试衡量效果

新鲜且未配置的数据库(2.2.5)，Ubuntu 14.04(VM).

Fresh and unconfigured database (2.2.5), Ubuntu 14.04 (VM).

结果:

$ ab -p post_loc.txt -T application/json -c 1 -n 10000 http://localhost:7474/db/data/node
This is ApacheBench, Version 2.3 <$Revision: 1604373 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking localhost (be patient)
Completed 1000 requests
Completed 2000 requests
Completed 3000 requests
Completed 4000 requests
Completed 5000 requests
Completed 6000 requests
Completed 7000 requests
Completed 8000 requests
Completed 9000 requests
Completed 10000 requests
Finished 10000 requests


Server Software:        Jetty(9.2.4.v20141103)
Server Hostname:        localhost
Server Port:            7474

Document Path:          /db/data/node
Document Length:        1245 bytes

Concurrency Level:      1
Time taken for tests:   14.082 seconds
Complete requests:      10000
Failed requests:        0
Total transferred:      14910000 bytes
Total body sent:        1460000
HTML transferred:       12450000 bytes
Requests per second:    710.13 [#/sec] (mean)
Time per request:       1.408 [ms] (mean)
Time per request:       1.408 [ms] (mean, across all concurrent requests)
Transfer rate:          1033.99 [Kbytes/sec] received
                        101.25 kb/s sent
                        1135.24 kb/s total

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.2      0      19
Processing:     1    1   1.3      1      53
Waiting:        0    1   1.2      1      53
Total:          1    1   1.3      1      54

Percentage of the requests served within a certain time (ms)
  50%      1
  66%      1
  75%      1
  80%      1
  90%      2
  95%      2
  98%      3
  99%      4
 100%     54 (longest request)

这个使用REST API创建10000个节点，在1个线程中没有属性.

This one creates 10000 nodes using REST API, with no properties in 1 thread.

如您所见，在Linux VM笔记本电脑上的事件具有默认设置-Neo4j能够在4毫秒或更短的时间内创建节点(99％).

As you can see, event on my laptop in Linux VM, with default settings - Neo4j is able to create nodes in 4ms or less (99%).

注意:我之前已经对数据库进行了预热(创建并删除了10万个节点).

Note: I have warmed up database before (created and deleted 100K nodes).

如果您正在寻找最佳Neo4j性能，则应遵循 Bolt 开发.这是Neo4j服务器的新二进制协议.

If you are looking for best Neo4j performance, you should follow Bolt development. This is new binary protocol for Neo4j server.

更多信息:此处， 查看全文