Logstash可以直接从远程日志中读取吗? [英] Can logstash read directly from remote log?
问题描述
我是Logtash的新手，几天以来我一直在阅读有关它的信息.像大多数人一样，我正在尝试建立一个集中式的日志记录系统，并将数据存储在elasticsearch中，然后使用kibana将数据可视化.我的应用程序已部署在许多服务器上，因此我需要从所有这些服务器中获取日志.在所有这些机器上安装logstash转发器并进行配置似乎是一件非常繁琐的任务(如果这是唯一的方法，我会这样做).有没有办法通过提及conf文件中某个地方的日志的URL而不是logstash转发器将logstash转发到logstash，来使logstash访问这些日志?仅供参考，我的应用程序已部署在tomcat上，并且可以通过URL http://:8080/application_name/error.log访问日志.
I am new to logstash and I am reading about it from couple of days. Like most of the people, I am trying to have a centralized logging system and store data in elasticsearch and later use kibana to visualize the data. My application is deployed in many servers and hence I need to fetch logs from all those servers. Installing logstash forwarder in all those machines and configuring them seems to be a very tedious task (I will do it if this is the only way). Is there a way for logstash to access those logs by mentioning the URL to logs somewhere in conf file instead of logstash forwarders forwarding it to logstash? FYI, my application is deployed on tomcat and the logs are accessible via URL http://:8080/application_name/error.log.
推荐答案
不是直接解决，而是有一些解决方法-这个想法是创建一个使用curl(或等效方法)来有效执行远程日志文件的"tail -f"，然后将运行结果输出到logstash.
Not directly but there are a few close workarounds - the idea is to create a program/script that will use curl (or it's equivalent) to effectively perform a "tail -f" of the remote log file, and the run that output into logstash.
这是一个可以解决问题的bash脚本:
Here's a bash script that does the trick:
url-tail.sh
此bash脚本监视url的更改并将其尾部打印到 标准输出.它充当"tail -f" linux命令.会有所帮助 用于可通过http访问的尾日志.
This bash script monitors url for changes and print its tail into standard output. It acts as "tail -f" linux command. It can be helpful for tailing logs that are accessible by http.
https://github.com/maksim07/url-tail
这里还有另一个类似的东西:
Another similar one is here:
https://gist.github.com/habibutsu/5420781
There are others out there, written in php or java: Tail a text file on a web server via HTTP
一旦您运行了该问题，便是如何将其放入logstash中-您可以:
Once you have that running the question is how to get it into logstash - you could:
-
将其放入stdin并使用stdin输入
Pipe it into stdin and use the stdin input
将其附加到文件中并使用文件输入
Have it append to a file and use the file input
使用管道输入本身运行命令并从命令行读取 命令的标准输出
Use the pipe input to run the command itself and read from the stdout of the command
细节决定成败，特别是使用logstash，因此您需要进行试验，但是这种方法应该对您有用.
The devil is in the details thought, particularly with logstash, so you'll need to experiment but this approach should work for you.
这篇关于Logstash可以直接从远程日志中读取吗?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
