在三台服务器上设置ELK堆栈的最佳方式 [英] Optimal way to set up ELK stack on three servers
问题描述
我正在寻找一个ELK堆栈，并有三个服务器这样做。虽然我已经找到了大量有关如何实际安装的文档和教程，并配置了elasticsearch，logstash和kibana，但是我发现有关如何在我的服务器上设置软件以最大限度地提高性能的信息较少。例如，最好在所有三个实例上设置elasticsearch，logstash和kibana，或者在第三个实例上安装弹性搜索，在第三个实例上安装弹性搜索，logstash和kibana？
与该问题相关联，如果我的集群中有多个弹性搜索服务器，我需要一个负载平衡器才能将请求传播给他们，还是可以将数据发送到一个服务器，并将相应地分发？
您的机器的大小也很重要。具有8GB RAM的三台机器与三个具有64GB或更多RAM的机器大不相同。
Kibana占用的资源很少。 Logstash更重CPU。弹性搜索是更多的RAM重。
使用弹性搜索集群，您通常需要每个分片的副本以进行冗余。这通常用两台服务器完成。如果您有第三个弹性搜索服务器，那么您将获得IO提升（将数据的两个副本写入三个服务器，从而降低负载）。此外，偶数个服务器可能会混淆主机哪一个，所以三个将有助于防止分裂脑问题。
这两个或三个节点将是数据节点，所以如果你向他们抛出查询或索引请求，他们可能需要将请求移动到不同的服务器（具有数据的服务器等）。请求还具有减少阶段，其中来自每个节点的数据在返回之前被组合。拥有一个较小的客户端节点 - 查询和索引请求就可以帮助您。当然，你会想要两个，使它们多余。
Logstash是最好的运行多线程，所以有多个可以专注的cpus是好的。拥有冗余/负载平衡的logstash机器也很好。 Kibana也可以在这些机器上运行。
所以，我们最多可以有7台机器。不是你想要听到的，对吧？
如果你坚定地限于3台机器，那么你想要像上面提到的那样运行所有三个弹性搜索。你需要在其余的鞋子上玩。
Logstash在两个上，一个是kibana？那么你有一个单一的失败的基地。
所有三个和所有三个日志记录的logstash如何？负载将分布在周围，所以希望每个服务器都会有一个小的增量。而且，如果机器足够强大，应该是可以的。
我在一个集群中运行logstash的机器，
$ b $一般建议是将1/2的系统RAM（高达〜31GB）分配给弹性搜索，将其余部分分配给操作系统。如果你要在同一台机器上运行logstash和kibana，那么你想降低这个值（可能是40％？），给logstash一些（15％？），然后将其余部分留给操作系统。
显然，您的机器的大小在这里很重要。
I am looking to set up an ELK stack and have three servers to do so. While I have found plenty of documentation and tutorials about how to actually install, and configure elasticsearch, logstash, and kibana, I have found less information about how I should set up the software across my servers to maximize performance. For example, would it be better to set up elasticsearch, logstash, and kibana on all three instances, or perhaps install elasticsearch on two instances and logstash and kibana on the third?
Related to that question, if i have multiple elasticsearch servers in my cluster, will I need a load balancer to spread requests to them, or can I send the data to one server, and it will distribute it accordingly?
The size of your machines would also be important. Three machines with 8GB of RAM is much different than three with 64GB or more...
Kibana takes very few resources. Logstash is more CPU-heavy. Elasticsearch is more RAM heavy.
With an elasticsearch cluster, you usually want a replica of each shard for redundancy. That's usually done with two servers. If you have a third elasticsearch server, then you'll get an IO boost (writing two copies of the data to three servers lowers the load). Also, an even number of servers can get confused as to which is the master, so three will help prevent "split brain" problems.
Those two or three nodes would be "data" nodes, so if you throw queries or indexing requests at them, they may need to move the request to a different server (the one with the data, etc). A request also has a "reduce" phase, where the data from each node is combined before being returned. Having a smaller "client" node - where queries and index requests go - helps with that. Of course, you'd want two, to make them redundant.
Logstash is best run multithreaded, so having multiple cpus that you can dedicate is nice. Having a redundant/load-balanced logstash machine is also nice. Kibana could run on these machines as well.
So, we're quickly up to 7 machines. Not what you wanted to hear, right?
If you're firmly limited to 3 machines, you'd want to run elasticsearch on all three as mentioned above. You need to shoehorn in the rest.
Logstash on two, kibana on one? Then you have a single point of failure for kibana.
How about logstash on all three and kibana on all three? The load would be distributed around, so hopefully would be a small increment for each server. And, if the machines are beefy enough, it should be OK.
I have machines in one cluster that run logstash,
The general recommendation is to allocate 1/2 the system RAM (up to ~31GB) to elasticsearch, leaving the rest to the operating system. If you were going to run logstash and kibana on the same machines, you'd want to lower that (to maybe 40%?), give logstash some (15%?) and leave the rest to the OS.
Clearly, the size of your machines is important here.
这篇关于在三台服务器上设置ELK堆栈的最佳方式的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
