为什么我的多线程插入比单线程插入性能更好? [英] Why does my multithreaded insert perform better than a single threaded insert?

问题描述

我调查了并发在MongoDB中，显然使用了数据库级别的锁定系统.我认为这意味着插入同一数据库中的多个线程的性能将比插入数据库中的单个线程的性能相似或更差.

I looked into concurrency in MongoDB, and apparently it uses a database level locking system. I thought that would mean that multiple threads inserting into the same database would perform similarly or worse than a single thread inserting into the database.

我发现当我有4个线程同时插入数据库时​​，性能几乎提高了一倍(以插入/秒为单位).

I found that when I got to 4 threads concurrently inserting into the database, performance nearly doubled (in terms of inserts/sec).

有什么理由可以使性能得到改善?我不明白为什么.

Is there any reason for why performance is getting better? I don't understand why.

如果有帮助，我有一个线程不断从服务器接收数据包并将其插入队列.我的4个线程不断从该队列中出队，并插入数据库中.

If it helps, I have one thread continually receiving packets from a server and inserting it into a queue. My 4 threads continually dequeue from that queue and insert into the database.

推荐答案

在MongoDB中发生写操作时，实际的写锁仅占完成时间的一小部分.

When a write happens in MongoDB the actual write lock is only held for a small fraction of the total time it takes to complete.

事实证明，写入RAM中的数据文件仅需几微秒(微秒，而不是毫秒).其余时间其他"内容仍在继续-这就是为什么如果您拥有多个线程或少数几个线程，则您将获得最佳性能的原因-在大多数情况下，限制因素将是诸如网络带宽或磁盘IO之类的限制或CPU(一旦您有足够的线程)，而不是锁.

As it turns out writing to data files in RAM only takes microseconds (micro, not milli). The rest of the time "other" stuff is going on - this is why you will get best performance if you have a lot more than one or a handful of threads - in most situations the limiting factor will be something like network bandwidth or disk IO or CPU (once you have enough threads) and not the lock.

如果有足够的线程在mongod处抛出写操作，并且文档非常小，则磁盘速度很快(用于将所有已写入的数据刷新到磁盘)，并且有足够的RAM来处理相关的索引更新，则数据库锁定可以成为写入吞吐量的限制因素，但通常不是成为瓶颈的第一件事或第二件事.

If you have enough threads throwing writes at mongod and the documents are very small, the disk is fast (for flushing all the written data to disk) and there is enough RAM for handling related index updates then the database lock can become the limiting factor for your write throughput but it's not normally the first or second thing that becomes the bottleneck.

我建议您使用mongostat，iostat和其他系统监视资源之类的工具来查看实际瓶颈在哪里.如果看不到任何内容，则说明您的应用程序无法以足够快的速度在数据库上进行工作，从而无法最大程度地发挥其容量.

I encourage you to use tools like mongostat, iostat, and other system monitoring resources to see where the actual bottlenecks are. If you don't see any then your application is not throwing work at the DB fast enough to max out its capacity.

这篇关于为什么我的多线程插入比单线程插入性能更好?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！