MongoDB-文件大小巨大且正在增长 [英] MongoDB - file size is huge and growing

问题描述

我有一个使用mongo来存储短期数据的应用程序.所有超过45分钟的数据都会被脚本删除，例如:

I have an application that use mongo for storing short living data. All data older than 45 minutes is removed by script something like:

oldSearches = [list of old searches]
connection = Connection()
db = connection.searchDB
res = db.results.remove{'search_id':{"$in":oldSearches}})

我已经检查了当前状态-

I've checked current status -

>db.results.stats()
{
        "ns" : "searchDB.results",
        "count" : 2865,
        "size" : 1003859656,
        "storageSize" : 29315124464,
        "nindexes" : 1,
        "ok" : 1
}

因此，根据此1gb数据占用29GB的存储空间.数据文件夹如下所示(您可能会看到很多文件很旧-最后一次访问可能是在5月中旬):

So, according to this 1gb of data occupies 29GB of storage. Data folder looks like this(You may see that many files are very old - last accessed in the middle of may):

ls -l /var/lib/mongodb/
total 31506556
-rwxr-xr-x 1 mongodb nogroup          6 2011-06-05 18:28 mongod.lock
-rw------- 1 mongodb nogroup   67108864 2011-05-13 17:45 searchDB.0
-rw------- 1 mongodb nogroup  134217728 2011-05-13 14:45 searchDB.1
-rw------- 1 mongodb nogroup 2146435072 2011-05-20 20:45 searchDB.10
-rw------- 1 mongodb nogroup 2146435072 2011-05-28 00:00 searchDB.11
-rw------- 1 mongodb nogroup 2146435072 2011-05-27 13:45 searchDB.12
-rw------- 1 mongodb nogroup 2146435072 2011-05-29 16:45 searchDB.13
-rw------- 1 mongodb nogroup 2146435072 2011-06-07 13:50 searchDB.14
-rw------- 1 mongodb nogroup 2146435072 2011-06-06 01:45 searchDB.15
-rw------- 1 mongodb nogroup 2146435072 2011-06-07 13:50 searchDB.16
-rw------- 1 mongodb nogroup 2146435072 2011-06-07 13:50 searchDB.17
-rw------- 1 mongodb nogroup 2146435072 2011-06-06 09:07 searchDB.18
-rw------- 1 mongodb nogroup  268435456 2011-05-13 14:45 searchDB.2
-rw------- 1 mongodb nogroup  536870912 2011-05-11 00:45 searchDB.3
-rw------- 1 mongodb nogroup 1073741824 2011-05-29 23:37 searchDB.4
-rw------- 1 mongodb nogroup 2146435072 2011-05-13 17:45 searchDB.5
-rw------- 1 mongodb nogroup 2146435072 2011-05-18 17:45 searchDB.6
-rw------- 1 mongodb nogroup 2146435072 2011-05-16 01:45 searchDB.7
-rw------- 1 mongodb nogroup 2146435072 2011-05-17 13:45 searchDB.8
-rw------- 1 mongodb nogroup 2146435072 2011-05-23 16:45 searchDB.9
-rw------- 1 mongodb nogroup   16777216 2011-06-07 13:50 searchDB.ns
-rw------- 1 mongodb nogroup   67108864 2011-04-23 18:51 test.0
-rw------- 1 mongodb nogroup   16777216 2011-04-23 18:51 test.ns

根据顶级" mongod使用29G的虚拟内存(和780Mb的RSS)

According to "top" mongod uses 29G of virtual memory ( and 780Mb of RSS)

为什么我会有这种异常值?我是否需要运行.remove()函数以外的其他东西来从旧值中清除数据库?

Why do i have such abnormal values? Do i need to run something additional to .remove() function to clean up database from old values?

推荐答案

虚拟内存大小和驻留大小对于mongod进程来说似乎很大.这是良性的:虚拟内存空间将大于打开和映射的数据文件的大小；驻留大小将因计算机上其他进程未使用的内存量而异.

Virtual memory size and resident size will appear to be very large for the mongod process. This is benign: virtual memory space will be just larger than the size of the datafiles open and mapped; resident size will vary depending on the amount of memory not used by other processes on the machine.

http://www.mongodb.org/display/DOCS/Caching

从MongoDB集合中删除对象时，不会自动垃圾回收该对象占用的空间，并且新记录只会追加到数据文件的末尾，从而使它们变得越来越大.这说明了一切:

When you remove an object from MongoDB collection, the space it occupied is not automatically garbage collected and new records are only appended to the end of data files, making them grow bigger and bigger. This explains it all:

http://www.mongodb.org/display/DOCS/Excessive+Disk+Space

对于初学者，只需使用:

For starters, simply use:

db.repairDatabase()

这篇关于MongoDB-文件大小巨大且正在增长的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！