ArangoDB打开的文件太多 [英] ArangoDB Too many open files

问题描述

几天以来，我们的ArangoDB安装遇到了问题.启动后几分钟/一小时不等，到数据库的所有连接都会被拒绝. arango日志文件显示打开的文件太多". "lsof | grep arango | wc -l"表明数据库具有大约50,000个打开文件句柄，这比最大文件句柄要低得多.由linux系统允许(大约3m). 有谁知道这个错误是从哪里来的?

since a few days we encounter a problem with our ArangoDB installation. A few minutes/up to an hour after start up all connections to the database are refused. The arango log file says that there are "Too many open files". A "lsof | grep arango | wc -l" shows that the database has around 50,000 open file handles, which is a lot under the max. allowed by the linux system (around 3m). Has anyone an idea where this error comes from?

我们正在使用具有3.13内核的Ubuntu Linux. 30 GB RAM和三个内核.该数据库仍然很小，大约有150万个条目，大小为50GB.

We are using a Ubuntu Linux with a 3.13 kernel. 30 GB RAM and three cores. The database is still very small with around 1,5m entries and a size of 50GB.

Thx，secana

Thx, secana

"netstat -anpt | fgrep 2480"显示:

"netstat -anpt | fgrep 2480" shows:

root@syssec-graphdb-001-test:~# netstat -anpt | fgrep 2480
tcp        0      0 10.215.17.193:2480      0.0.0.0:*               LISTEN               7741/arangod
tcp        0      0 10.215.17.193:2480      10.215.50.30:53453      ESTABLISHED          7741/arangod
tcp        0      0 10.215.17.193:2480      10.215.50.31:49299      ESTABLISHED          7741/arangod
tcp        0      0 10.215.17.193:2480      10.215.50.30:53155      ESTABLISHED          7741/arangod

"ulimit -n"的结果为1024，所以我认为〜50,000都是arango进程在一起.

"ulimit -n" has a result of 1024, so I think that the ~50,000 are all arango processes together.

数据库失效前日志文件中的最后几行:

Last lines in log file before the database died:

2015-05-26T12:20:43Z [9672] ERROR cannot open datafile '/data/arangodb/databases/database-235999516/collection-28464454696/datafile-18806474509149.db': 'Too many open files'
2015-05-26T12:20:43Z [9672] ERROR cannot open datafile '/data/arangodb/databases/database-235999516/collection-28464454696/datafile-18806474509149.db': Too many open files
2015-05-26T12:20:43Z [9672] DEBUG [arangod/VocBase/collection.cpp:1632] cannot open '/data/arangodb/databases/database-235999516/collection-28464454696', check failed
2015-05-26T12:20:43Z [9672] ERROR cannot open document collection from path '/data/arangodb/databases/database-235999516/collection-28464454696'

推荐答案

增加最大值似乎很有意义.允许进程管理的打开文件数.考虑到规定的数据库大小约为50 GB，(默认值)1024的值似乎太低.

It looks like it will make sense to increase the max. number of open files a process is allowed to manage. Given the stated database size of around 50 GB, the (presumably default) value of 1024 seems to be too low.

arangod 对于每个并行客户端连接都将需要一个文件描述符.可能并不很多，但是面对HTTP保持活动连接，这可能已经说明了几个文件描述符.

arangod will require one file descriptor for each parallel client connection. That may not be many, but in the face of HTTP keep-alive connections this could already account for several file descriptors.

此外，活动集合的每个数据文件都需要进行内存映射，并且也需要一个文件描述符.默认数据文件大小为32 MB，数据库大小为50 GB(在磁盘上)将已经使用1,600个文件描述符:

Additionally, each datafile of an active collection will need to be memory-mapped and cost one file descriptor as well. With the default datafile size of 32 MB, a database size of 50 GB (on disk) will already consume 1,600 file descriptors:

50 GB database size / (32 MB default size / 1 datafile) = 1600 datafiles

为 arangod 用户和环境增加ulimit -n值将是有意义的.您可以通过以--server.descriptors-minimum <value>选项启动它来确认 arangod 实际上可以使用配置数量的文件描述符.

Increasing the ulimit -n value for the arangod user and environment therefore will make sense. You can confirm that arangod can actually use the configured number of file descriptors by starting it with option --server.descriptors-minimum <value>, e.g.

--server.descriptors-minimum 32768 

表示那么多文件描述符.如果 arangod 无法有效使用指定数量的文件描述符，它将在启动时失败并出现致命错误.当然，该选项也可以放在arangod.conf文件中.

for that many file descriptors. If arangod cannot effectively use that specified amount of file descriptors, it will fail at start with a fatal error. Of course that option can also be put into the arangod.conf file.

此外，可以通过集合的journalSize参数增加(新)数据文件的默认大小.这目前无济于事，但会减少将来保存的数据所需的文件描述符的数量.

Additionally, the default size for (new) datafiles can be increased via the journalSize parameter for collections. That won't help right now, but will lower the number of required file descriptors for data saved in the future.

这篇关于ArangoDB打开的文件太多的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！