Logstash输出Elasticsearch在AWS EC2 [英] Logstash output to Elasticsearch on AWS EC2

问题描述

我无法配置logstash输出到AWS EC2的Elasticsearch集群。

I'm having trouble configuring logstash to output to an Elasticsearch cluster on AWS EC2.

我使用Logstash版本1.1.5和Elasticsearch 1.19.8。

I'm using Logstash version 1.1.5 and Elasticsearch 1.19.8.

这是我的输出配置，logstash：

This is my output configuration in logstash:

output {
  stdout { debug => true debug_format => "json"}
  elasticsearch {
    cluster => "logstash-searcher"
    node_name => "logstash-indexer"
  }
}

这是elasticsearch.yml相应的配置。

and this is the corresponding configuration in elasticsearch.yml

cluster.name: logstash-searcher
path.data: /usr/local/elasticsearch/data
path.work: /usr/local/elasticsearch/tmp
path.logs: /usr/local/elasticsearch/logs
path.plugins: /usr/local/elasticsearch/plugins
bootstrap.mlockall: true
cloud.aws.region: eu-west-1
cloud.aws.access_key: --
cloud.aws.secret_key: --
discovery.type: ec2
discovery.ec2.host_type: public_ip
discovery.ec2.groups: elasticsearch
gateway.type: s3
gateway.s3.bucket: es-logstash

transport.tcp.port: 9300-9400

我开始logstash使用：

I start logstash using:

java -jar logstash-1.1.5-monolithic.jar agent -f shipper.conf

和启动的一段时间后，我得到这些失败：

And after a while of startup I get these failures:

Failed to index an event, will retry {:exception=>org.elasticsearch.discovery.MasterNotDiscoveredException: waited for [1m], 

我怀疑是logstash需要使用类似云AWS为其elasticsearch客户能够找到集群。有没有人对AWS的作品为例配置？

My suspicion is that logstash needs to use something like the cloud-aws for its elasticsearch client to be able to find the cluster. Does anyone have an example configuration that works on aws?

推荐答案

问题是，logstash的嵌入式elasticsearch实例是使用它的默认搜索模式。由于elasticsearch群集配置云AWS logstash的嵌入式elasticsearch需要为好。

The problem is that the embedded elasticsearch instance of logstash was using its default discovery mode. Since the elasticsearch cluster is configured with cloud-aws the embedded elasticsearch of logstash needs to as well.

要做到这一点，你必须一个elasticsearch.yml配置文件添加到logstash的工作目录。您还需要通过将其添加到类路径中，以提供云AWS插件。

To do that you have to add an elasticsearch.yml configuration file to the working directory of logstash. You also need to supply the cloud-aws plugin by adding it to the class path.

java -cp logstash-1.1.7-monolithic.jar:cloud-aws/* logstash.runner agent -f shipper.conf

使用这个配置我设法让logstash输出到我的elasticsearch集群。

Using this configuration I managed to get logstash to output to my elasticsearch cluster.

这篇关于Logstash输出Elasticsearch在AWS EC2的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！