多个AWS EC2实例之间共享存储 [英] Shared storage between multiple AWS EC2 instances

问题描述

我现在面临一个问题，共享多个EC2实例之间的存储。我将不得不运行繁重的工作，所以我会需要大量的实例来做到这一点。 一方面，我有一个EBS卷附加到一个服务器实例。在另一边，我有一个工人实例。我创造了这个工人实例的AMI，然后我创造了这个AMI的几个实例副本。还有在同一VPC所有正在运行。基本上，服务器实例发送作业，工人正在执行的任务。 我想救我时，工人们正在运行的作业一些日志文件，在共享存储是这样的：

I'm facing a problem to share a storage between multiple EC2 instances. I'm going to have to run heavy jobs so I'll need a lot of instances to do it. On one side ,I have an EBS volume attached to one server instance. On the other side I have a a worker instance. I created an AMI of this worker instance and then I created several instances copies of this AMI. There are all running on the same VPC. Basically the server instance is sending jobs and the workers are executing the job. I would like to save some log files when my workers are running the jobs, in the share storage something like:

worker_1 / LOGFILE.LOG

worker_1/ logfile.log

worker_2 / LOGFILE.LOG

worker_2/ logfile.log

有什么能做到这一点的最好的解决办法？

What could be the best solution to do that?

• 在我读了它不可能在同一 EBS卷连接到多个实例。
• 我看了一下 GlusterFS 但这里是我发现了什么：

• I read it's not possible to attach the same EBS volume to multiple instances.
• I had a look at GlusterFS but here is what I found:

以前实现概念验证有两台服务器，在不同的可用区域，复制EBS卷与EXT4文件系统，我们将列出其中，GlusterFS不应该使用的情况： 顺序文件从多个服务器同时写入例如日志。锁定系统可能会导致严重的问题，如果你GlusterFS中存储日志。理想的解决方案是存储在本地，然后使用S3存档他们。如果需要，我们可以将多个服务器日志之前或之后将它们存储在S3。的

"Before realizing a proof of concept with two servers, in different availability zones, replicating an EBS volume with an ext4 filesystem, we will list the cases where GlusterFS should not be used: Sequential files written simultaneously from multiple servers such as logs. The locking system can lead to serious problems if you store logs within GlusterFS. The ideal solution it’s to store them locally then use S3 to archive them. If necessary we can consolidate multiple server logs before or after storing them in S3."

• 最后，我还检查 S3斗安装与 s3fs 但是我发现这不是一个很好的选择太多：

• And finally, I've also checked S3 bucket mounted with s3fs but I found out it's not a good option too:

你不能部分更新与s3fs因此更改一个字节将重新上传整个文件的文件。然后，如果你想使小的增量变化那么它一定没有。不能使用s3fs - S3只是不工作的方式，你不能增量更改文件的

那么这可能是一个很好的解决了我的问题，让我的工人写在一个共享的存储其日志文件？

Then what could be a good solution to my problems and allows my workers to write their log files in a share storage?

感谢您的帮助！

Romanzo

推荐答案

感谢您的答案。但最后我使用NFS的实例之间，它的工作原理pretty的好！

Thanks for the answers. but finally I'm using NFS between the instances and it works pretty well!

这篇关于多个AWS EC2实例之间共享存储的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！