使用 Postman 模拟 ELK Beat 输出到 Logstash [英] Mock an ELK Beat output to Logstash with Postman

问题描述

我有将数据发送到 logstash 的 Elastic Beats(如 metricbeat 和 auditbeat).出于测试目的，我想模拟一些从节拍到 logstash 输入的数据.

I have Elastic Beats (like metricbeat and auditbeat) that send data to logstash. For testing purposes, I'd like to mock some data from the beats to the logstash input.

我正在使用 beats 中的 logstash 输出插件(yaml 配置文件):

I'm using the logstash output plugin in the beats (yaml config file):

output.logstash:
  enabled: true
  hosts: ["127.0.0.1:5044"]

以及 logstash 中的 beats 输入插件(.conf 管道配置文件):

and the beats input plugin in logstash (.conf pipeline config file):

input {
    beats {
        port => "5044"
    }
}

我天真地假设 beat 是通过 HTTP 请求与 logstash 通信，所以我使用 Postman 向 logstash 发出 POST 请求，但我在 logstash 中收到 InvalidFrameProtocolException: InvalidFrameProtocolException: Invalid version of beats protocol 错误.

I naïvely assumed the beats were communicating with logstash via HTTP requests so I used Postman to make a POST request to logstash, but I got an InvalidFrameProtocolException: Invalid version of beats protocol error in logstash.

我尝试使用wireshark检查数据包以查看原始数据包的格式，但我只能看到TCP数据包而没有HTTP协议.我猜这些插件不通过 HTTP 请求进行通信.

I tried to inspect the packets with wireshark to see how the original packets are formatted but all I could see was TCP packets and no HTTP protocol. I guess then that the plugins don't communicate via HTTP requests.

有更多专业知识的人可以对此发表评论吗?有没有办法通过邮递员模拟节拍输出数据到logstash?

Can someone with more expertise comment on this ? Is there a way to mock beats output data to logstash via postman ?

推荐答案

感谢@leandrojmp，我知道beats和logstash通过lumberjack协议而不是HTTP进行通信.

Thanks to @leandrojmp I know the beats and logstash communicate via the lumberjack protocol and not HTTP.

我最终使用了 http logstash 输入插件，这样我就可以通过邮递员模拟我的节拍事件.虽然我没有修改我原来的管道配置文件，但是我使用了logstash的能力来加载几个管道配置文件.所以我用我的 http 插件和一些过滤器创建了另一个 logstash 管道配置文件，以删除 http 特定数据，如header".字段:

I ended up using the http logstash input plugin so that I can mock my beats events via postman. I didn't modify my original pipeline configuration file though, but I used the ability of logstash to load several pipeline configuration files. So I created another logstash pipeline configuration file with my http plugin and some filter to remove the http specific data like the "header" field:

input {
    stdin { }
    http {
        port => "8081"
    }
}

filter {
    mutate {
        remove_field => ["headers"]
    }
}

然后，当我想启动 logstash 进行测试时，我可以加载这两个文件，方法是将它们放在同一目录中:

I can then load the two files when I want to launch logstash for testing, by placing them in the same directory:

bin/logstash -f logstash.d/*.conf

或者当我不测试时只是原始配置文件:

Or just the original config file when I'm not testing:

bin/logstash -f logstash.d/pipeline.conf

这篇关于使用 Postman 模拟 ELK Beat 输出到 Logstash的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！