MONGODB Mongoimport可能加快速度吗? [英] MONGODB Mongoimport possible speed up?

问题描述

我有一个2 GB的CSV文件，其中包含9 M条记录，这些记录是使用本机mongoimport工具导入到MongoDB中的.它以每秒8K的速率导入CSV.总耗时为10分钟.导入的速度是相当合理的，但它似乎比MySQL LOAD DATA INFILE版本慢得多(仅需2分钟即可将所有记录插入数据库).虽然这是可以接受的(MongoDB是为JSON类型的对象构建的，并且加速通常是在查询中而不是在插入中进行)，但我想知道是否有某种方法可以加快MongoDB mongoimport中每秒完成的插入次数? 我只有一台具有8 GB RAM和4核的计算机. 谢谢.

I have a 2 GB CSV file with 9 M records that I import into MongoDB using the native mongoimport tool. It imports the CSV at a rate of 8K per second. The overall time taken is 10 minutes. The speed of import is quite reasonable, but it seems to be much slower than the MySQL LOAD DATA INFILE version(takes only 2 minutes to insert all of the records into the database). While this is acceptable (MongoDB is built for JSON type objects and speed ups are generally in querying and not in inserting), I would like to know if there is some way that I can speed up the number of inserts per second done in MongoDB mongoimport? I have only one computer with 8 GB RAM and 4 cores. Thanks.

推荐答案

由于大部分时间都可能花费在将JSON对象序列化为BSON(本地MongoDB格式)上，因此，如果您可以拆分文件并拥有几个并行的作业，每个作业都运行mongoimport并带有单独的文件.

Since the majority of time is likely spent serializing JSON objects into BSON (native MongoDB format) you will likely get faster import if you can split up your file and have several parallel jobs each running mongoimport with a separate file.

这篇关于MONGODB Mongoimport可能加快速度吗?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！