Linux中的Node.js和打开文件限制 [英] Node.js and open files limit in linux
问题描述
我运行一个node.js客户端，该客户端向我的服务器发送很多请求(该请求也写在node.js中).
服务器收到对特定文件的请求，并将其上传到S3.
两分钟后，我收到以下错误消息，一切正常:
{ [NetworkingError: getaddrinfo ENOTFOUND]
message: 'getaddrinfo ENOTFOUND',
code: 'NetworkingError',
errno: 'ENOTFOUND',
syscall: 'getaddrinfo',
region: 'us-east-1',
hostname: 'XXXX.s3.amazonaws.com',
retryable: true,
time: Sun Oct 12 2014 11:27:54 GMT-0400 (EDT),
_willRetry: false }
经过一番小小的研究，我发现它发生了可能是因为我我试图打开太多的文件句柄或套接字，而这实际上只能在一段时间后发生.
但是据我了解，node.js应该为我封装这个问题.换句话说，node.js应该了解文件描述符的限制，并根据该限制打开一个新的描述符.这是基于事件使用一个用户线程的优点(我在哪里错了?)
如果node.js不这样做，那么针对此错误的最佳解决方案是不要增加我的打开文件限制(这是一个糟糕的主意，因为我们需要在此计算机上拥有良好的性能. ，我如何确定如果我增加这个数字，这个错误不会再次出现?我怎么知道操作系统应该为此应用程序指定的数字?)
在Ubuntu中，默认的打开文件描述符为1024. 您可以使用
从终端设置ulimit -nulimit -n #a number
，但这只会在当前登录会话上进行更改.要使更改永久生效，请使用以下命令.
ulimit -n #see the number of open files
sudo vi /etc/security/limits.conf #open the file in vi
现在同时设置两个:
用户软文件9000
用户硬nofile 65000
root soft nofile 9000
root hard nofile 65000
sudo vi /etc/pam.d/common-session
现在添加此行:
会话必填pam_limits.so
http://posidev.com/blog/2009/06/04/set-ulimit-parameters-on-ubuntu/
关于如何在不弄乱ulimit的情况下克服此问题，则必须将触发请求的总数限制为小于当前的ulimit -n.
通过创建Http长期连接池并开始通过它们推送请求，或者限制从应用程序触发的最大并发请求数，任何超出此限制的请求都将它们添加到队列中，以便稍后进行处理.这只是一个概念，如果您需要更多有关此问题的帮助，请向我们显示一些代码，以便我们可以想象您要尝试执行的操作.
希望这会有所帮助.
I run an node.js client which send a lot of requests to my server (which written also in node.js).
The server get a request for a specific file, and upload it to S3.
I get the following error after two minutes that everything goes well:
{ [NetworkingError: getaddrinfo ENOTFOUND]
message: 'getaddrinfo ENOTFOUND',
code: 'NetworkingError',
errno: 'ENOTFOUND',
syscall: 'getaddrinfo',
region: 'us-east-1',
hostname: 'XXXX.s3.amazonaws.com',
retryable: true,
time: Sun Oct 12 2014 11:27:54 GMT-0400 (EDT),
_willRetry: false }
After I did a small research, i found that it happens probably because I'm trying to open too many file handles, or sockets, which can really only happen after a while.
But as I understand, node.js should encapsulate this issue for me. In other words, node.js should understand the limit of the file descriptor, and open a new one according to this limit. This is the advantages of using one user thread based on event (Where I wrong?)
If node.js doesn't do it, what the best solution for this error, which is not to increase my open file limit (This is a bad idea, because we need a good performance on this machine. This and more, How will I be sure that if I would increase the number, this error would not appear again? How would I know the number the OS should have for this application?)
The default open file descriptors are 1024 in Ubuntu. you can set the ulimit -n from the terminal using
ulimit -n #a number
but this will make the changes on the current login session only. to make the changes permanent use these commands.
ulimit -n #see the number of open files
sudo vi /etc/security/limits.conf #open the file in vi
now set both:
user soft nofile 9000
user hard nofile 65000
root soft nofile 9000
root hard nofile 65000
sudo vi /etc/pam.d/common-session
now add this line:
session required pam_limits.so
http://posidev.com/blog/2009/06/04/set-ulimit-parameters-on-ubuntu/
EDIT:
About how to get over this problem without messing with the ulimits then you will have to limit the total number of the fired requests to be less than your current ulimit -n.
by either creating a pool of Http long lived connections and start push your requests through them, or you limit the max concurrent requests fired from your app and any requests exceeding this limit add them to a queue so you can process them later.
this is just a concept if you want more help with this problem, show us some code so we can imagine what you are trying to do.
Hope this helped.
这篇关于Linux中的Node.js和打开文件限制的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
