从输入文件路径logstash中提取字段? [英] Extracting fields from input file path logstash?

问题描述

我想从各种目录中读取我的日志文件，例如:Server1, Server2...

I want to read my log files from various directories, like: Server1, Server2...

Server1 有子目录 cron, auth...在这些子目录中分别是 log 文件.

Server1 has subdirectories as cron, auth...inside these subdirectories is the log file respectively.

所以我正在考虑阅读这样的文件:

So I am contemplating of reading files like this:

input{
         file{            
                                     #path/to/folders/server1/cronLog/cron_log

                             path => "path/to/folders/**/*_log"
            }
    }

但是，我很难过滤它们，即知道对于哪个服务器 (Server1) 和日志类型 (cron)，我必须应用 grok 模式:

However, I am having difficulty in filtering them i.e to know that for which server (Server1) and logtype (cron), I must apply the grok pattern:

例如:我想做这样的事情

Eg: I thought of doing something like this

if [path] =~ "auth"{

grok{
            match => ["message", ***patteren****]
        }
    }else if [path] =~ "cron"{
        grok{
            match => ["message", ***pattern***]
        }

上面的 cron 是日志文件(不是 cronLog 目录).但像这样我也想过滤 server name 因为每个服务器都会有 cron、auth 等日志.

Above cron is for log file (not cronLog directory). But like this I also want to filter on server name as every server will have cron, auth,etc logs.

如何过滤两者?

有没有办法从输入中的 path 中获取目录名称??喜欢从这里

Is there a way to grab directory names from path in input ?? Like from here

path => "path/to/folders/**/*_log"

我应该如何进行?任何帮助表示赞赏?

How should I proceed? Any help is appreciated?

推荐答案

它非常直截了当，几乎与我的其他答案完全一样......你使用路径上的 grok 提取出你关心的部分然后你可以从那里做任何你想做的事情

its very straight forward, and almost exactly like in my other answer... you use the grok on the path to extract the pieces out that you care about and then you can do whatever you want from there

filter {
   grok { "path", "path/to/here/(?<server>[^/]+)/(?<logtype>[^/]+)/(?<logname>.*) }
   if [server] == "blah" && [logtype] =~ "cron" {
       grok { "message", "** pattern **" }
   }
}

这篇关于从输入文件路径logstash中提取字段?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！