如何稳定是s3fs安装一个Amazon S3存储作为一个本地目录 [英] How stable is s3fs to mount an Amazon S3 bucket as a local directory

问题描述

如何稳定是s3fs安装一个Amazon S3存储在linux的本地目录？它是推荐/稳定高要求的生产环境？

How stable is s3fs to mount an Amazon S3 bucket as a local directory in linux? Is it recommended/stable for high demand production environments?

有没有更好的/类似的解决方案？

Are there any better/similar solutions?

更新：会不会更好使用EBS，并通过NFS挂载到其它所有的AMI

Update: Would it be better to use EBS and to mount it via NFS to all other AMIs?

推荐答案

有一个很好的文章上s3fs这里，其中阅读后，我使出了EBS分享。

There's a good article on s3fs here, which after reading I resorted to an EBS Share.

它突出使用s3fs，即关系到S3的固有局限性，当一些重要的注意事项：

It highlights a few important considerations when using s3fs, namely related to the inherent limitations of S3:

• 在没有文件可以超过5GB
• 您不能部分更新文件，以便改变一个字节将重新上传整个文件。
• 在许多小文件的操作是非常有效的（它们都是独立的S3对象，毕竟），但大的文件是非常低效
• 虽然S3支持部分/分块下载，s3fs不利用这一点，所以如果你想读一个1GB的文件只是一个字节，你必须下载整个GB。

因此​​，它取决于你存储什么s3fs是否是一个可行的选择。如果你存储的发言权，照片，要编写整个文件或整个文件读从来没有增量改变文件，那么它的精致，虽然有人会问，如果你这样做，那么为什么不直接使用S3的API直接？

It therefore depends on what you are storing whether s3fs is a feasible option. If you're storing say, photos, where you want to write an entire file or read an entire file never incrementally change a file, then its fine, although one may ask, if you're doing this, then why not just use S3's API Directly?

如果你谈论蒋云良数据，（比如数据库文件，日志文件），你想使小的增量变化那么它绝对不能 - S3只是不工作的方式，你不能增量更改文件

If you're talking about appliation data, (say database files, logging files) where you want to make small incremental change then its a definite no - S3 Just doesn't work that way you can't incrementally change a file.

上述文章并谈了类似的应用程序 - s3backer - 这得到周围的性能问题通过实施虚拟文件系统在S3。这得到周围的性能问题，但是本身有自己的几个问题：

The article mentioned above does talk about a similar application - s3backer - which gets around the performance issues by implementing a virtual filesystem over S3. This gets around the performance issues but itself has a few issues of its own:

• 在高风险的数据损坏，由于延迟写入
• 在过小的块大小（例如，4K默认值）可以添加显著 额外费用（例如，$ 130，50GB与4K块价值的存储设备）
• 在过大的块大小可以添加显著数据传输和存储 费。
• 在内存使用量可以让人望而却步：在默认情况下它会缓存1000块
  。 使用默认4K块大小，这不是一个问题，但大多数用户
  可能需要增加块大小。

• High risk for data corruption, due to the delayed writes
• too small block sizes (e.g., the 4K default) can add significant extra costs (e.g., $130 for 50GB with 4K blocks worth of storage)
• too large block sizes can add significant data transfer and storage fees.
• memory usage can be prohibitive: by default it caches 1000 blocks.
  With the default 4K block size that's not an issue but most users
  will probably want to increase block size.

我使出EBS安装掘进从EC2实例共享。但是你应该知道，虽然在最高效的选择它有一个很大的问题 安装NFS共享EBS有它自己的问题 - 一个单点故障;如果机器多数民众赞成共享EBS卷出现故障，那么你输在哪访问该共享中的所有计算机进行访问。

I resorted to EBS Mounted Drived shared from an EC2 instance. But you should know that although the most performant option it has one big problem An EBS Mounted NFS Share has its own problems - a single point of failure; if the machine that's sharing the EBS Volume goes down then you lose access on all machines which access the share.

这是一个风险，我能够住在一起，是我选择在最后的选项。我希望这有助于。

This is a risk I was able to live with and was the option I chose in the end. I hope this helps.

这篇关于如何稳定是s3fs安装一个Amazon S3存储作为一个本地目录的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！