在Logstash中解析JSON [英] Parsing JSON in Logstash
问题描述
我有一些日志文件是由 Log4J2 生成的.我正在使用log4j2.xml配置文件中的JSONLayout将日志输出到.json文件中.我的JSONLayout的定义如下:
I have some log files generated by Log4J2. I am outputting the logs into a .json file using the JSONLayout in the log4j2.xml configuration file. My JSONLayout is defined like this:
<JSONLayout complete="false"></JSONLayout>
当日志被输入到我的计算机上的日志文件中时,它们将一个接一个地追加,并在logs.log中如下所示:
As logs get entered into the log file on my machine, they are appended, one-after-another, and look like this in logs.log:
{
"logger":"com.mycompany.myLogger",
"timestamp":"1396792374326",
"level":"ERROR",
"thread":"pool-2-thread-2",
"message":"System: unable to perform action",
"throwable":"java.lang.NullPointerException\\n\tat com.myCompany.MyClass $.java:432)\\n\tat java.lang.Thread.run(Thread.java:744)\\n"
},
我正在尝试构造此JSON,以便可以从ElasticSearch查询它.在此过程中,我试图将自定义字段添加到所有记录.为此,我正在使用以下内容:
I am trying to structure this JSON so that I can query it from ElasticSearch. During that process, I'm trying to add a custom field to ALL records. To do that, I'm using the following:
input {
file {
type => "json"
path => "/var/logs/myApp/logs.log"
}
}
filter {
json {
add_tag => [ "HardcodedTagName"]
source => "message"
}
}
output {
elasticsearch {
protocol => "http"
codec => "json"
host => "[myServerAddress]"
port => "9200"
}
}
奇怪的是,似乎从未添加我的自定义标签.同时,我真的很想将JSON分解为可以在ElasticSearch中查询的字段.我想查询的东西很明显.他们是:
Oddly, my custom tag NEVER seems to get added. At the same time, I'd really like to break out my JSON into fields that I can query in ElasticSearch. The things I'd like to query are clearly available. They are:
- 级别
- 消息
- 时间戳
但是,我似乎无法获得这些信息.当我在Kibana中看到JSON记录时,我只会看到以下内容:
Yet, I can't seem to get this information out. When I see the JSON records in Kibana, I just see something like this:
{
"_index": "logstash-2014.04.07",
"_type": "json",
"_id": "tG-s6-5pSnq5HZwLhM6Dxg",
"_score": null,
"_source": {
"message": " \"message\":\"System: unable to perform action\"\r",
"@version": "1",
"@timestamp": "2014-04-07T18:01:38.739Z",
"type": "json",
"host": "ip-MyipAddress",
"path": "/var/logs/myApp/logs.log"
},
"sort": [
1396893698739,
1396893698739
]
}
最让我惊讶的是,日志级别甚至没有.线程信息也不是.令我惊讶的是,我找不到使用过滤器的Log4J示例的博客文章.衷心感谢您的帮助!
What surprises me most is that the log level isn't even there. Neither is the thread information. I was surprised I couldn't find a blog post of a Log4J example that uses a filter. Any help is sincerely appreciated!
推荐答案
您不需要使用json过滤器.您可以在输入时指定 编解码器 格式.
You do not need to use the json filter. You can specify the codec format at your input.
input {
file {
type => "json"
path => "/var/logs/myApp/logs.log"
codec => json
}
}
Logstash将自动将您的日志解析为json格式并添加相应的字段.
Logstash will automatically parse your log to json format and add the corresponding field.
这篇关于在Logstash中解析JSON的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!