如何在单个AWS EC2实例上设置3节点Elasticsearch集群? [英] How to setup a 3-node Elasticsearch cluster on a single AWS EC2 instance?

问题描述

我目前正在尝试使用docker-compose文件在单个 EC2实例(即仅使用一个实例)上部署3节点Elasticsearch集群.问题是我无法让3个节点相互通信以形成集群.

I am currently trying to deploy a 3-node Elasticsearch cluster on a single EC2 instance (i.e. using ONE instance only) using a docker-compose file. The problem is I could not get the 3 nodes to communicate with each other to form the cluster.

在Windows 10计算机上，我使用了官方的Elasticsearch:6.4.3映像，而对于AWS EC2，我使用的是自定义的Elasticsearch:6.4.3映像，其中安装了 ec2-discovery插件使用" docker build -t mdasri/eswithec2disc."命令进行构建.请参阅下面的dockerfile.

On my Windows 10 machine, I used the official Elasticsearch:6.4.3 image while for AWS EC2, I am using a custom Elasticsearch:6.4.3 image with ec2-discovery plugin installed where I build using the "docker build -t mdasri/eswithec2disc ." command. Refer to dockerfile below.

dockerfile:

FROM docker.elastic.co/elasticsearch/elasticsearch:6.4.3

RUN /usr/share/elasticsearch/bin/elasticsearch-plugin install --batch discovery-ec2

我成功地在Windows 10计算机上使用docker-compose在本地设置了3节点Elasticsearch集群.在我的docker-compose文件中，我有3个不同的Elasticsearch服务来组成3个节点:es01，es02，es03.我希望使用相同的docker-compose文件在AWS EC2实例上设置集群，但遇到错误.

I was successful in setting up the 3-node Elasticsearch cluster locally using docker-compose on my Windows 10 machine. In my docker-compose file, I have 3 different Elasticsearch services to make up the 3-nodes: es01, es02, es03. I was hoping to use the same docker-compose file to set up the cluster on AWS EC2 instance but I was hit with error.

我正在使用" ecs-cli compose -f docker-compose.yml up "命令部署到AWS EC2. ecs-cli compose的状态为:已启动容器...".

I am using the "ecs-cli compose -f docker-compose.yml up" command to deploy to AWS EC2. The status of the ecs-cli compose was: "Started container...".

因此要检查群集状态，我键入了 x.x.x.x/_cluster/health?pretty ，但是遇到了以下错误:

So to check the cluster status, I typed x.x.x.x/_cluster/health?pretty, but was hit with this error:

{
  "error" : {
    "root_cause" : [
      {
        "type" : "master_not_discovered_exception",
        "reason" : null
      }
    ],
    "type" : "master_not_discovered_exception",
    "reason" : null
  },
  "status" : 503
}

当我在ssh进入后评估EC2实例中的每个docker容器日志时，这是我在 ALL 3个容器中遇到的错误:

When I assess each of the docker container logs in the EC2 instance after I ssh in, this is the error I face in ALL 3 containers:

[2019-06-24T06:19:43,880] [WARN] [o.e.d.z.UnicastZenPing] [es01] 无法解析主机[es02]

[2019-06-24T06:19:43,880][WARN ][o.e.d.z.UnicastZenPing ] [es01] failed to resolve host [es02]

这是我的对应AWS EC2服务的docker-compose文件:

version: '2'
services:

es01:
image: mdasri/eswithec2disc
container_name: es01
cpu_shares: 100
mem_limit: 2147482548

ulimits:
  memlock:
    soft: -1
    hard: -1
  nofile:
    soft: 65536
    hard: 65536

ports:
  - "9200:9200"
  - "9300:9300"

environment:
  - "cluster.name=aws-cluster"
  - "node.name=es01"
  - "node.master=true"
  - "node.data=false"
  - "discovery.zen.hosts_provider=ec2"
  - "discovery.zen.ping.unicast.hosts=es01, es02"
  - "discovery.zen.minimum_master_nodes=2"
  - "ES_JAVA_OPTS= -Xmx256m -Xms256m"
  - "bootstrap.memory_lock=true"

volumes:
  - /usr/share/elasticsearch/data

networks:
  - esnet

es02:
image: mdasri/eswithec2disc
container_name: es02
cpu_shares: 100
mem_limit: 2147482548

ulimits:
  memlock:
    soft: -1
    hard: -1
  nofile:
    soft: 65536
    hard: 65536

environment:
  - "cluster.name=aws-cluster"
  - "node.name=es02"
  - "node.master=true"
  - "node.data=false"
  - "discovery.zen.hosts_provider=ec2"
  - "discovery.zen.ping.unicast.hosts=es01, es02"
  - "ES_JAVA_OPTS= -Xmx256m -Xms256m"
  - "bootstrap.memory_lock=true"

volumes:
  - /usr/share/elasticsearch/data

networks:
  - esnet
es03:
image: mdasri/eswithec2disc
container_name: es03
cpu_shares: 100
mem_limit: 2147482548

ulimits:
  memlock:
    soft: -1
    hard: -1
  nofile:
    soft: 65536
    hard: 65536

environment:
  - "cluster.name=aws-cluster"
  - "node.name=es03"
  - "node.master=false"
  - "node.data=true"
  - "discovery.zen.hosts_provider=ec2"
  - "discovery.zen.ping.unicast.hosts=es01,es02"
  - "ES_JAVA_OPTS= -Xmx256m -Xms256m"
  - "bootstrap.memory_lock=true"

volumes:
  - /usr/share/elasticsearch/data

networks:
  - esnet

networks:
  esnet:

请帮助我，因为在过去的1-2周中我一直在解决此问题. 附注:请让我知道你们还需要其他哪些信息.谢谢！

Please help me as I've been stuck on this problem for the past 1-2 weeks. P.S: Please let me know what other information do you guys need. Thanks!

推荐答案

您需要在docker-compose中配置links以便能够解析:

you need to configure links in your docker-compose to be able to resolvable:

来自docker-compose Docs:

from docker-compose Docs:

链接到另一个服务中的容器.既可以指定服务名称，也可以指定链接别名(SERVICE:ALIAS)，也可以仅指定服务名称.

Link to containers in another service. Either specify both the service name and a link alias (SERVICE:ALIAS), or just the service name.

web:
  links:
   - db
   - db:database
   - redis

，也请参见@ Mishi.Srivastava

and see the comment also from @Mishi.Srivastava

这篇关于如何在单个AWS EC2实例上设置3节点Elasticsearch集群?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！