Logstash-无法打开< file_path>没有权限 [英] Logstash - Failed to open <file_path> Permission denied

问题描述

我正在使用logstash将所有文本日志从存储推送到弹性搜索. 我的存储大小约为1 TB.首先，我已经开始将368 GB数据(可能是几十万个文件)推入弹性搜索，但是logstash失败并出现以下错误.

I am using logstash to push all the text logs from storage to elastic search. My storage size is about 1 TB. To Start with I have started to push 368 GB data (may be few hundred thousand files) to elastic search but logstash is failing with following error.

{:timestamp=>"2014-05-15T00:41:12.436000-0700", :message=>"/root/share/archive_data/sessionLogs/965c6f46-1a5e-4820-a68d-7c32886972fc/Log.txt: file grew, old size 0, new size 1557420", :level=>:debug, :file=>"filewatch/watch.rb", :line=>"81"}
{:timestamp=>"2014-05-15T00:41:12.437000-0700", :message=>":modify for /root/share/archive_data/sessionLogs/965c6f46-1a5e-4820-a68d-7c32886972fc/Log.txt, does not exist in @files", :level=>:debug, :file=>"filewatch/tail.rb", :line=>"77"}
{:timestamp=>"2014-05-15T00:41:12.441000-0700", :message=>"_open_file: /root/share/archive_data/sessionLogs/965c6f46-1a5e-4820-a68d-7c32886972fc/Log.txt: opening", :level=>:debug, :file=>"filewatch/tail.rb", :line=>"98"}
{:timestamp=>"2014-05-15T00:41:12.441000-0700", :message=>"(warn supressed) failed to open /root/share/archive_data/sessionLogs/965c6f46-1a5e-4820-a68d-7c32886972fc/Log.txt: Permission denied - /root/share/archive_data/sessionLogs/965c6f46-1a5e-4820-a68d-7c32886972fc/Log.txt", :level=>:debug, :file=>"filewatch/tail.rb", :line=>"110"}

共享已通过网络安装.我正在使用root用户启动logstash.用户应该具有安装所需的所有访问权限. 共享目录具有以下访问权限 drwxr-xr-x 44根根0 May 13 08:36分享

share is network mounted. I am using root user to start logstash. User should have all the access needed to mount. share directory has following access drwxr-xr-x 44 root root 0 May 13 08:36 share

现在，我的日志文件是静态的，它们不会更改.

Now, my log files are static they don't change.

所以，我的问题是-有没有要让logstash知道一旦处理一个日志文件就不存储文件句柄.我认为发生上述错误是因为日志文件的数量很大.

So, my question is - Is there anyway to let logstash know that do not store file handles once it process one log file. I think above error is occurred because number of log files is huge.

我已经提交了 bug ，并且logstash中存在现有错误，该错误表示当日志文件数量更多时，logstash效果不佳.

I have already filed a bug and there is existing bug in logstash which says that logstash doesn't do well when log files are more in number.

我在这里看到一些重复的问题，但是我想知道是否有人对这种问题有任何经验?

I see some duplicate issues here but I would like to know if anybody has any experience with this kind of issue?

推荐答案

我认为，对于logstash 1.4.2，唯一的答案是:

I think, for logstash 1.4.2, the only answer is to:

• 从受监视的目录中移动或删除文件
• 重新启动日志记录

我认为没有其他方法可以让Logstash发布文件句柄处理已处理且不会再添加的日志.

I don't think there's any other way to have logstash release file handles of logs that have been processed and won't be added to any more.

这篇关于Logstash-无法打开< file_path>没有权限的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！