将日志从ECS Fargate发送到Elasticsearch的最佳方法 [英] Best approach for sending logs from ECS Fargate into Elasticsearch

问题描述

我们有一个设置，其中有多个容器运行在AWS ECS Fargate中部署的NodeJS服务（node：11-alpine docker image）。

We have a setup with multiple containers running NodeJS services(node:11-alpine docker image) deployed in AWS ECS Fargate.

我们已经有一个正在运行的ElasticSearch实例从非Fargate应用程序收集日志。我想将日志从Fargate容器传递到此ElasticSearch实例中，但是我很难找出最好的方法。

We already have a running ElasticSearch instance collecting logs from non-Fargate application. I would like to pass the logs from the Fargate containers into this ElasticSearch instance, but I have a hard time to figure out what is the best approach.

1）
似乎一种方法是从Cloudwatch-> Lambda-> ElasticSearch流日志。似乎有点矫kill过正-没有其他方法可以做到这一点吗？

1) It seem one way is to stream the logs from Cloudwatch --> Lambda --> ElasticSearch. It seem a bit overkill - isn't there another way to do this?

2）
我希望我可以运行一个可以收集的Logstash docker实例容器中的日志，但是我不确定运行Fargate时是否可行？

2) I was hoping i could run a Logstash docker instance that could collect the logs from the containers but I am not sure if this is possible when running Fargate?

3）
我应该在每个容器上安装FileBeat之类的东西，

3) Should I install something like FileBeat on each container and let that send the logs?

感谢您的帮助。

推荐答案

1）似乎一种方法是从Cloudwatch-> Lambda-> ElasticSearch流日志。似乎有点矫kill过正-没有其他方法可以做到这一点吗？

1) It seems one way is to stream the logs from Cloudwatch --> Lambda --> ElasticSearch. It seem a bit overkill - isn't there another way to do this?

如果您正在寻找基于AWS的托管解决方案，这是正确的方法。您实际上并不需要编写Lambda函数，AWS可以为您完成此功能，而且这似乎更具矫健性，因为这是一种更具管理性和面向AWS的方法。另外，此解决方案的作用与Logstash相同。

If you're looking for an AWS based managed solution, that is the right way. You don't really need to write a Lambda function, AWS does it for you and that doesn't seem like overkill as this is more managed and more AWS oriented approach. Also, what this solution is going to do for is going to be the same as Logstash.

2）我希望我可以运行Logstash docker实例可以从容器中收集日志，但是我不确定在运行Fargate时是否可以这样做？

2) I was hoping i could run a Logstash docker instance that could collect the logs from the containers but I am not sure if this is possible when running Fargate?

是的，是可能的。

3）我应该在每个容器上安装FileBeat之类的东西并发送日志吗？

3) Should I install something like FileBeat on each container and let that send the logs?

您可以根据需要使用FileBeat，Fluentd，FunctionBeat或Logstash。

You can use FileBeat, Fluentd, FunctionBeat or Logstash as you like.

注意：如果您想运行自己的Logstash容器，请不要启用CloudWatch日志记录，也不要使用它。但是我建议为此使用基于AWS的解决方案。

Note: If you're thinking of running your own Logstash container, don't enable CloudWatch logging as well as you're not going to use that either. But I would recommend going for AWS based solution for that.

这篇关于将日志从ECS Fargate发送到Elasticsearch的最佳方法的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！