Logstash打开的文件太多？ [英] Logstash too many files opened?

问题描述

我正在运行 logstash 1.4.1 并运行它，我在控制台上得到关注：

I am running logstash 1.4.1 and on running it I am getting following on the console:

    failed to open /home/Desktop/Input/2014-11-03/abc.log: Permission denied - /home/Desktop/Input/2014-11-03/abc.log {:level=>:warn}
    failed to open /home/Desktop/Input/2014-10-14/abc.log: Permission denied - /home/Desktop/Input/2014-10-14/abc.log {:level=>:warn}
    failed to open /home/Desktop/2014-10-16/abc.log: Permission denied - /home/Desktop/Input/2014-10-16/abc.log {:level=>:warn}

..............

    log4j, [2014-11-13T16:20:04.583]  WARN: org.elasticsearch.discovery.zen: [logstash-HP-Pro-3330-MT-7573-11818] failed to connect to master [[Kyle Gibney][eM0ZmUkAQpaXd6_n4JWjzA][-HP-Pro-3330-MT][inet[/xx.xx..xx..xx:9300]]], retrying...
    org.elasticsearch.transport.ConnectTransportException: [Kyle Gibney][inet[/xx.xx.xx.xx:9300]] General node connection failure
        at org.elasticsearch.transport.netty.NettyTransport.connectToNode(NettyTransport.java:668)
        at org.elasticsearch.transport.netty.NettyTransport.connectToNode(NettyTransport.java:615)
        at org.elasticsearch.transport.TransportService.connectToNode(TransportService.java:129)
        at org.elasticsearch.discovery.zen.ZenDiscovery.innerJoinCluster(ZenDiscovery.java:338)
        at org.elasticsearch.discovery.zen.ZenDiscovery.access$500(ZenDiscovery.java:79)
        at org.elasticsearch.discovery.zen.ZenDiscovery$1.run(ZenDiscovery.java:286)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:744)
    Caused by: org.elasticsearch.common.netty.channel.ChannelException: Failed to open a socket.
        at org.elasticsearch.common.netty.channel.socket.nio.NioClientSocketChannel.newSocket(NioClientSocketChannel.java:43)
        at org.elasticsearch.common.netty.channel.socket.nio.NioClientSocketChannel.<init>(NioClientSocketChannel.java:82)
        at org.elasticsearch.common.netty.channel.socket.nio.NioClientSocketChannelFactory.newChannel(NioClientSocketChannelFactory.java:212)
        at org.elasticsearch.common.netty.channel.socket.nio.NioClientSocketChannelFactory.newChannel(NioClientSocketChannelFactory.java:82)
        at org.elasticsearch.common.netty.bootstrap.ClientBootstrap.connect(ClientBootstrap.java:212)
        at org.elasticsearch.common.netty.bootstrap.ClientBootstrap.connect(ClientBootstrap.java:182)
        at org.elasticsearch.transport.netty.NettyTransport.connectToChannels(NettyTransport.java:702)
        at org.elasticsearch.transport.netty.NettyTransport.connectToNode(NettyTransport.java:647)
        ... 8 more
    Caused by: java.net.SocketException: Too many open files
        at sun.nio.ch.Net.socket0(Native Method)
        at sun.nio.ch.Net.socket(Net.java:423)
        at sun.nio.ch.Net.socket(Net.java:416)
        at sun.nio.ch.SocketChannelImpl.<init>(SocketChannelImpl.java:104)
        at sun.nio.ch.SelectorProviderImpl.openSocketChannel(SelectorProviderImpl.java:60)
        at java.nio.channels.SocketChannel.open(SocketChannel.java:142)
        at org.elasticsearch.common.netty.channel.socket.nio.NioClientSocketChannel.newSocket(NioClientSocketChannel.java:41)
        ... 15 more

所以logstash打开文件达到一定的阈值，之后给予该阈值以后文件的许可被拒绝，并且打开了太多的文件。

So logstash is opening files upto a certain threshold and after that giving permission denied for the files after that threshold and saying too many files opened.

任何想法？

推荐答案

你需要检查你的linux用户的文件句柄的极限是多少。您可以检查的地方是：

You need to check what's the limit of your file handle for your linux user. One place you could check is:

cat /etc/security/limits.conf

您定义硬和软限制的位置。另一种方法是使用ulimit命令，如下所示：

Where you define hard and soft limit. Another way would be to use ulimit command like below:

ulimit -a | grep nofiles ##Or ulimit -n

这篇关于Logstash打开的文件太多？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！