Error when configuring logstash to output to an Elasticsearch cluster on AWS EC2

Question

I'm facing trouble when configuring logstash to output to an Elasticsearch cluster on AWS EC2. Logstash version 1.1.5 and Elasticsearch 1.19.8.

Output configuration in logstash:

cluster.name: logstash-searcher
path.data: /usr/local/elasticsearch/data
path.work: /usr/local/elasticsearch/tmp
path.logs: /usr/local/elasticsearch/logs
path.plugins: /usr/local/elasticsearch/plugins
bootstrap.mlockall: true
cloud.aws.region: eu-west-1
cloud.aws.access_key: --
cloud.aws.secret_key: --
discovery.type: ec2
discovery.ec2.host_type: public_ip
discovery.ec2.groups: elasticsearch
gateway.type: s3
gateway.s3.bucket: es-logstash

transport.tcp.port: 9300-9400

Getting an error of sort

Failed to index an event, will retry {:exception=>org.elasticsearch.discovery.MasterNotDiscoveredException: waited for [1m], 

So, what could be the issue?

Answer 1

Ok. The issue could be that embedded elasticsearch instance of logstash was using its default discovery mode. Because the elasticsearch cluster is configured with cloud-aws the embedded elasticsearch of logstash needs to as well.

So, add an elasticsearch.yml configuration file to the working directory of logstash