您如何将Spring Boot日志直接摄取到Elastic中 [英] How do you ingest Spring boot logs directly into elastic

问题描述

我正在研究将Spring Boot应用程序日志直接发送到弹性搜索的可行性.不使用文件拍或logstash.我相信Ingest插件可能对此有所帮助.

I’m investigating feasability of sending spring boot application logs directly into elastic search. Without using filebeats or logstash. I believe the Ingest plugin may help with this.

我最初的想法是使用基于TCP的登录进行此操作.

My initial thoughts are to do this using logback over TCP.

https://github.com/logstash/logstash-logback-encoder

<?xml version="1.0" encoding="UTF-8"?>
<configuration>
  <appender name="stash" class="net.logstash.logback.appender.LogstashTcpSocketAppender">
      <destination>127.0.0.1:4560</destination>
      <encoder class="net.logstash.logback.encoder.LogstashEncoder" />
  </appender>

  <root level="DEBUG">
      <appender-ref ref="stash" />
  </root>
</configuration>

因此，通过上述内容，您可以将日志直接发送到logstash中.我只是想知道是否有可能使用较新的摄取功能并跳过使用logstash吗?通过使用摄取方法通过网络将json编码的日志直接发送到Elastic中?

So looking at the above you can send logs directly into logstash. Im just wondering if it was possible to use the newer functionality of ingest and skip using logstash? By sending json encoded logs directly into elastic over the network using the ingest method?

https://www.elastic.co/blog /new-way-to-ingest-part-1

我的问题

我想知道是否可行?如果可以的话，您可以解释一下您将如何做. 还有什么可能的陷阱等?

I’m wondering if this is possible? If so could you explain how you would do it. Also what possible what would be the pitfalls etc.

推荐答案

我刚刚尝试了我的建议，但效果很好.

I just tried my suggestion and it worked out perfectly.

首先，在您的POM中添加此依赖项:

First, add this dependency in your POM:

    <dependency>
        <groupId>org.logback-extensions</groupId>
        <artifactId>logback-ext-loggly</artifactId>
        <version>0.1.2</version>
    </dependency>

然后，在您的logback.xml配置中，添加一个附加程序和一个记录器，如下所示:

Then, in your logback.xml configuration, add an appender and a logger like this:

<appender name="ES" class="ch.qos.logback.ext.loggly.LogglyAppender">
    <endpointUrl>http://localhost:9200/tests/test?pipeline=logback</endpointUrl>
    <pattern>%m</pattern>
</appender>
<logger name="es" level="INFO" additivity="false">
    <appender-ref ref="ES"/>
</logger>

您还需要定义这样的提取管道:

You also need to define an ingest pipeline like this:

PUT _ingest/pipeline/logback
{
  "description": "logback pipeline",
  "processors": [
    {
      "set" : {
        "field": "source",
        "value": "logback"
      }
    }
  ]
}

然后，在您的代码中，您可以使用该记录器并将所需的任何数据发送到ES

Then, in your code you can use that logger and send whatever data you have to your ES

private Logger esLogger = LoggerFactory.getLogger("es");
...
esLogger.info("{\"message\": \"Hello World from Logback!\"}");

此文档将最终出现在您的ES中:

And this document will end up in your ES:

{
    "_index": "tests",
    "_type": "test",
    "_id": "AV3Psj5MF_PW7ho1yJhQ",
    "_score": 1,
    "_source": {
      "source": "logback",
      "message": "Hello World from Logback!",
    }
}

这篇关于您如何将Spring Boot日志直接摄取到Elastic中的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！