获取CloudTrail登录到Logstash [英] Getting CloudTrail Logs into Logstash
问题描述
我想从ClouldTrail登录到ElasticSearch这样我们就可以看到什么是我们的AWS回事占比较好。
I am trying to get logs from ClouldTrail into ElasticSearch so that we can see what is going on in our AWS account better.
我已经建立了两个Logstash和ElasticSearch我的机器(Ubuntu的14.04)上,并且可以将文本从标准输入
来ElasticSearch。然而,当我尝试使用S3输入没有被添加到ElasticSearch。
I have set up both Logstash and ElasticSearch on my machine (Ubuntu 14.04), and can push text from stdin
to ElasticSearch. However when I try to use the S3 input nothing is added to ElasticSearch.
下面是即时通讯使用conf文件,我已经打消了我的亚马逊键
Here is the conf file Im using, I have removed my amazon keys
input {
s3 {
bucket => 'ko-cloudtrail-log-bucket'
secret_access_key => ''
access_key_id => ''
delete => false
interval => '60'
region => 'eu-west-1'
type => 'CloudTrail'
codec => cloudtrail {}
}
}
output {
stdout {}
elasticsearch {
host => '127.0.0.1'
}
}
我安装 logstash- codeC- cloudtrail codeC但文档是pretty的稀疏。
I have install the logstash-codec-cloudtrail codec but the documentation is pretty sparse.
我与-v并没有什么运行Logstash即使在我的终端没有错误打印到标准输出
。有我丢失的东西?
I get no errors in my terminal even when running Logstash with -v and nothing is printed to stdout
. Is there something I am missing?
推荐答案
下面是我的cloudtrail输入。它有一个轻微的发出─它重复记录的伟大工程。由于 preFIX
表示,我把cloudtrail日志在S3:// bucketname / cloudtrail,不是根
Here's my cloudtrail input. It works great with one minor issue- it duplicates records. As the prefix
indicates, I put the cloudtrail logs at s3://bucketname/cloudtrail, not the root.
的突变是可选的即可。该EventSource的突变是使日志更易读一点,和红宝石ingest_time给了我一个日期记录在ELK-出现了,否则,它只有事件的时间。最后,我放弃,只是增加了噪音,我的系统中一个非常普遍的纪录。
The mutations are optional. The eventSource mutation is to make the logs a little more readable, and the ruby ingest_time gives me a date the record showed up in ELK- otherwise, it only has the time of the event. Finally, I drop a very common record that just adds noise to my system.
input {
s3 {
bucket => "bucketname"
delete => false
interval => 60 # seconds
prefix => "cloudtrail/"
type => "cloudtrail"
codec => "cloudtrail"
credentials => "/etc/logstash/s3_credentials.ini"
sincedb_path => "/opt/logstash_cloudtrail/sincedb"
}
}
filter {
if [type] == "cloudtrail" {
mutate {
gsub => [ "eventSource", "\.amazonaws\.com$", "" ]
add_field => {
"document_id" => "%{eventID}"
}
}
if ! [ingest_time] {
ruby {
code => "event['ingest_time'] = Time.now.utc.strftime '%FT%TZ'"
}
}
if [eventSource] == "elasticloadbalancing" and [eventName] == "describeInstanceHealth" and [userIdentity.userName] == "secret_username" {
drop {}
}
}
}
该credentials.ini格式的 S3输入页面解释;它只是这样的:
The credentials.ini format is explained on the s3 input page; it's just this:
AWS_ACCESS_KEY_ID=
AWS_SECRET_ACCESS_KEY=
这篇关于获取CloudTrail登录到Logstash的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!