如何从消息字段中删除尾随换行符 [英] How to remove trailing newline from message field

问题描述

我将带有Logstash的Glassfish 4日志文件运送到ElasticSearch接收器.如何使用Logstash从消息字段中删除结尾的换行符?

I am shipping Glassfish 4 logfiles with Logstash to an ElasticSearch sink. How can I remove with Logstash the trailing newline from a message field?

我的活动如下:

{
  "@timestamp" => "2013-11-21T13:29:33.081Z",
  "message" => "[2013-11-21T13:29:32.577+0000] [glassfish 4.0] [INFO] [] [javax.resourceadapter.mqjmsra.lifecycle] [tid: _ThreadID=142 _ThreadName=Thread-43] [timeMillis: 1385040572577] [levelValue: 800] [[\n  MQJMSRA_RA1101: GlassFish MQ JMS Resource Adapter stopped.]]\n",
  "@version" => "1",
  "tags" => ["multiline", "date_filtered"],
  "host" => "myhost",
  "path" => "../server.log"
} 

推荐答案

您必须使用具有正确模式的多行过滤器，以告诉logstash，每个具有空格的行都属于该行.将此行添加到您的conf文件中.

You have to use the multiline filter with the correct pattern, to tell logstash, that every line with precending whitespace belongs to the line before. Add this lines to your conf file.

filter{
  ...
  multiline {
    type => "gflogs"
    pattern => "\[\#\|\d{4}"
    negate => true
    what => "previous"
  }
  ...
}

您还可以包括grok插件来处理时间戳并从索引的蜜蜂中过滤掉不规则的行.

You can also include grok plugin to handle timestamp and filter irregular lines from beeing indexed.

查看同一台计算机上具有单个logstash实例的完整堆栈

See complete stack with single logstash instance on same machine

input {
  stdin {
    type => "stdin-type"
  }

  file {
    path => "/path/to/glassfish/logs/*.log"
    type => "gflogs"
  }
}

filter{
  multiline {
    type => "gflogs"
    pattern => "\[\#\|\d{4}"
    negate => true
    what => "previous"
  }

  grok {
    type => "gflogs"
    pattern => "(?m)\[\#\|%{TIMESTAMP_ISO8601:timestamp}\|%{LOGLEVEL:loglevel}\|%{DATA:server_version}\|%{JAVACLASS:category}\|%{DATA:kv}\|%{DATA:message}\|\#\]"
    named_captures_only => true
    singles => true
  }

  date {
    type => "gflogs"
    match => [ "timestamp", "ISO8601" ]
  }

  kv {
    type => "gflogs"
    exclude_tags => "_grokparsefailure"
    source => "kv"
    field_split => ";"
    value_split => "="
  }
}

output {
  stdout { codec => rubydebug }
  elasticsearch { embedded => true }
}

这对我有用.请在 logstash-usergroup 上查看此帖子.我还可以为您提供最新的出色的 logstash书.这也是支持logstash作者工作的好方法.

This worked for me. Pleas look also this post on logstash-usergroup. I can also advice the great and up to date logstash book. Its also a good way to support the work of the logstash author.

希望在任何JUG-Berlin活动中见到您！

Hope to see you on any JUG-Berlin Event!

这篇关于如何从消息字段中删除尾随换行符的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！