节点和错误：EMFILE，打开的文件太多 [英] node and Error: EMFILE, too many open files
问题描述
有些日子我一直在搜索错误的工作解决方案
For some days I have searched for a working solution to an error
错误：EMFILE，打开文件太多
似乎很多人都有同样的问题。通常的答案是增加文件描述符的数量。所以，我试过这个：
It seems that many people have the same problem. The usual answer involves increasing the number of file descriptors. So, I've tried this:
sysctl -w kern.maxfiles = 20480 ，
默认值是10240.这在我看来有点奇怪，因为我在目录中处理的文件数量低于10240.即使是陌生人，我仍然会收到同样的错误我增加了文件描述符的数量。
The default value is 10240. This is a little strange in my eyes, because the number of files I'm handling in the directory is under 10240. Even stranger, I still receive the same error after I've increased the number of file descriptors.
第二个问题：
经过多次搜索后我找到解决太多打开文件问题的方法：
After a number of searches I found a work around for the "too many open files" problem:
var requestBatches = {};
function batchingReadFile(filename, callback) {
// First check to see if there is already a batch
if (requestBatches.hasOwnProperty(filename)) {
requestBatches[filename].push(callback);
return;
}
// Otherwise start a new one and make a real request
var batch = requestBatches[filename] = [callback];
FS.readFile(filename, onRealRead);
// Flush out the batch on complete
function onRealRead() {
delete requestBatches[filename];
for (var i = 0, l = batch.length; i < l; i++) {
batch[i].apply(null, arguments);
}
}
}
function printFile(file){
console.log(file);
}
dir = "/Users/xaver/Downloads/xaver/xxx/xxx/"
var files = fs.readdirSync(dir);
for (i in files){
filename = dir + files[i];
console.log(filename);
batchingReadFile(filename, printFile);
不幸的是我仍然收到同样的错误。
这段代码有什么问题？
Unfortunately I still recieve the same error. What is wrong with this code?
最后一个问题（我是javascript和节点的新手），我正在开发一个网络
应用程序，每天约有5000个用户请求。我在使用python和java等其他语言进行
编程方面有多年的经验。所以最初我想用django或play框架来开发这个应用程序。然后我发现了节点，我必须说非阻塞I / O模型的想法非常好，诱人，而且最重要的是非常快！
One last question (I'm new to javascript and node), I'm in the process of developping a web application with a lot of requests for about 5000 daily users. I've many years of experience in programming with other languages like python and java. so originally I thought to developp this application with django or play framework. Then I discovered node and I must say that the idea of non-blocking I/O model is really nice, seductive, and most of all very fast!
但是什么样的我应该期望与节点有什么问题？它是经过生产验证的Web服务器吗？你有什么经历？
But what kind of problems should I expect with node? Is it a production proven web server? What are your experiences?
推荐答案
当 graceful-fs 不起作用......或者您只是想了解泄漏的来源。按照这个过程。
For when graceful-fs doesn't work... or you just want to understand where the leak is coming from. Follow this process.
（例如，如果你的问题是套接字，那么graceful-fs就不会修理你的旅行车。）
(e.g. graceful-fs isn't gonna fix your wagon if your issue is with sockets.)
此命令将输出nodejs进程的打开句柄数：
This command will output the number of open handles for nodejs processes:
lsof -i -n -P | grep nodejs
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
...
nodejs 12211 root 1012u IPv4 151317015 0t0 TCP 10.101.42.209:40371->54.236.3.170:80 (ESTABLISHED)
nodejs 12211 root 1013u IPv4 151279902 0t0 TCP 10.101.42.209:43656->54.236.3.172:80 (ESTABLISHED)
nodejs 12211 root 1014u IPv4 151317016 0t0 TCP 10.101.42.209:34450->54.236.3.168:80 (ESTABLISHED)
nodejs 12211 root 1015u IPv4 151289728 0t0 TCP 10.101.42.209:52691->54.236.3.173:80 (ESTABLISHED)
nodejs 12211 root 1016u IPv4 151305607 0t0 TCP 10.101.42.209:47707->54.236.3.172:80 (ESTABLISHED)
nodejs 12211 root 1017u IPv4 151289730 0t0 TCP 10.101.42.209:45423->54.236.3.171:80 (ESTABLISHED)
nodejs 12211 root 1018u IPv4 151289731 0t0 TCP 10.101.42.209:36090->54.236.3.170:80 (ESTABLISHED)
nodejs 12211 root 1019u IPv4 151314874 0t0 TCP 10.101.42.209:49176->54.236.3.172:80 (ESTABLISHED)
nodejs 12211 root 1020u IPv4 151289768 0t0 TCP 10.101.42.209:45427->54.236.3.171:80 (ESTABLISHED)
nodejs 12211 root 1021u IPv4 151289769 0t0 TCP 10.101.42.209:36094->54.236.3.170:80 (ESTABLISHED)
nodejs 12211 root 1022u IPv4 151279903 0t0 TCP 10.101.42.209:43836->54.236.3.171:80 (ESTABLISHED)
nodejs 12211 root 1023u IPv4 151281403 0t0 TCP 10.101.42.209:43930->54.236.3.172:80 (ESTABLISHED)
....
请注意：1023u（最后一行） - 这是第1024个文件句柄，这是默认的最大值。
Notice the: 1023u (last line) - that's the 1024th file handle which is the default maximum.
现在，看看最后一栏。这表明哪个资源是开放的。您可能会看到许多行都具有相同的资源名称。希望现在告诉您在代码中查找泄漏的位置。
Now, Look at the last column. That indicates which resource is open. You'll probably see a number of lines all with the same resource name. Hopefully, that now tells you where to look in your code for the leak.
如果您不知道多个节点进程，请首先查找哪个进程有pid 12211。我会告诉你这个过程。
If you don't know multiple node processes, first lookup which process has pid 12211. That'll tell you the process.
在上面的例子中，我注意到有一堆非常相似的IP地址。它们都是 54.236.3。### 通过ip地址查找，我能够确定它与pubnub相关。
In my case above, I noticed that there were a bunch of very similar IP Addresses. They were all 54.236.3.### By doing ip address lookups, was able to determine in my case it was pubnub related.
使用此语法确定进程已打开的句柄数...
Use this syntax to determine how many open handles a process has open...
我使用此命令测试在我的应用中执行各种事件后打开的文件数。
I used this command to test the number of files that were opened after doing various events in my app.
lsof -i -n -P | grep8465| wc -l </ code>
lsof -i -n -P | grep "8465" | wc -l
# lsof -i -n -P | grep "nodejs.*8465" | wc -l
28
# lsof -i -n -P | grep "nodejs.*8465" | wc -l
31
# lsof -i -n -P | grep "nodejs.*8465" | wc -l
34
您的流程限制是多少？
ulimit -a
行想要看起来像这样：
打开文件（-n）1024
The line you want will look like this:
open files (-n) 1024
- 在Ubuntu 14.04上测试，nodejs v.7.9
如果您希望打开许多连接（websockets是一个很好的例子），您可以永久性地增加限制：
In case if you are expecting to open many connections (websockets is a good example), you can permanently increase the limit:
-
file：/etc/pam.d/common-session （添加到最后）
session required pam_limits.so
文件： /etc/security/limits.conf （添加到最后，或编辑，如果已存在）
root soft nofile 40000
root hard nofile 100000
重启你的nodejs并注销/从ssh登录。
restart your nodejs and logout/login from ssh.
这篇关于节点和错误：EMFILE，打开的文件太多的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
