Neo4j 在高事务并发下变得相当慢 [英] Neo4j get pretty slower with high transaction concurrency
问题描述
我使用的是 neo4j 3.1.0 企业版.我的图中的主要逻辑是:有IP"节点和用户"节点，并且都具有UNIQUE"约束.每次用户登录时，我都会添加从 IP 到用户的关系.
I was using neo4j 3.1.0 enterprise edition. The main logical in my graph is: There are "IP" nodes and "User" nodes and both have "UNIQUE" constraints. Each time an user login, I add a relationship from IP to User.
这是我的插入密码:
MERGE (i:IP {ip:"1.2.3.4"})
MERGE (u:User {username:"xxx@gmail.com"})
MERGE (i) - [l:SUCC] -> (u)
SET i:ExpireNode, i.expire={expire}
SET u:ExpireNode, u.expire={expire}
SET l.expire={expire}, l.login={login}
插入速度非常快.但是当节点数增长到数百万时，变得非常缓慢，有时插入节点和关系需要1秒以上.
The insert is pretty fast as the beginning. But when the number of node grows to millions, it became very slow and sometimes took more than 1 second to insert nodes and relationships.
我该如何优化它?我使用 12 核 CPU 和 64G 内存运行 neo4j.初始头大小为 16G，页缓存为 30G.
How could I optimize it? I was running neo4j with 12-cores CPU and 64G memory. The initial head size is 16G and page cache is 30G.
--------------------------------------------------------------
--------------------------------------------------------------
在 Web UI 中测试了相同的密码，命令花费了 10 毫秒.但是使用java驱动，有时会超过1s.下面是我的java代码:
Tested the same cypher in Web UI and it took 10ms for commands. But by using java driver, it will sometimes take more than 1s. Below is my java code:
try (Transaction tx = session.beginTransaction()) {
for (Login login : loginList) {
Value value = login2Operation(login);
tx.run(INSERT_COMMANDS_SUCC, value);
}
tx.success();
}
--------------------------------------------------------------
--------------------------------------------------------------
经过一番探索，我发现如果在5个线程中运行，插入速度会显着提高.但是整体速度太慢，我不得不增加到100个线程.然后每个速度的单次插入增长到1s.所以，我认为问题是因为 Neo4j 的并行能力.
After some exploring, I found that the insert speed increased significantly if running in 5 threads. But the overall speed is too slow and I have to increase to 100 threads. Then the single insert of each speed grows to 1s. So, I believe the problem is because the parallel ability of Neo4j.
在 neo4j.conf 中，我添加了 dbms.threads.worker_count=200.但它没有帮助.有什么想法吗?
In the neo4j.conf, I added dbms.threads.worker_count=200. But it's not helping. Any ideas?
推荐答案
感谢@InverseFalcon 的建议，UNWIND 操作帮助很大！
Thanks to @InverseFalcon 's advices and the UNWIND operation helps a lot!
这篇关于Neo4j 在高事务并发下变得相当慢的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
