Amazon S3的架构 [英] Amazon S3 architecture

问题描述

但后@ <一href="http://highscalability.com/amazon-architecture">http://highscalability.com/amazon-architecture解释一般亚马逊的架构，我想知道如何Amazon S3的实现。

While the post @ http://highscalability.com/amazon-architecture explains Amazon's architecture in general, I am interested in knowing how Amazon S3 is implemented.

我的一些猜测是

1. 在像HDFS分布式文件系统 <一href="http://hadoop.apache.org/core/docs/current/hdfs_design.html">http://hadoop.apache.org/core/docs/current/hdfs_design.html
2. 在一个非关系型数据库持久喜欢的CouchDB http://couchdb.apache.org/

1. A distributed file system like HDFS http://hadoop.apache.org/core/docs/current/hdfs_design.html
2. A non relational persistent DB like CouchDB http://couchdb.apache.org/

时，才有可能实现类似于此的使用脚本语言如Python或PHP的规模要小得多的东西？

Is it be possible to implement something similar to this on a much smaller scale using scripting languages like Python or PHP?

推荐答案

亚马逊S3采用在迪纳摩论文中描述的体系结构实现：

Amazon S3 is implemented using the architecture described in the Dynamo Paper:

<一个href="http://www.allthingsdistributed.com/2007/10/amazons_dynamo.html">http://www.allthingsdistributed.com/2007/10/amazons_dynamo.html

本文介绍了一致性哈希，以及如何和为什么的保障是最终一致性。

The paper explains consistent hashing, and how and why the guarantee is "eventual consistency".

冲突解决他们谈论迪纳摩不会暴露在S3的用户。它在内部使用亚马逊的应用程序，但对于S3，唯一的解决冲突是最后一次写入获胜。

The conflict resolution they talk about for Dynamo is not exposed to users of S3. It is used internally in Amazon's applications, but for S3, the only conflict resolution is last write wins.

编辑：沃纳VOGELS有说。迪纳摩不直接暴露在外作为Web服务;但是，迪纳摩和类似亚马逊的技术用于功率器件我们的亚马逊Web服务，如S3 <一href="http://www.allthingsdistributed.com/2007/10/amazons_dynamo.html">http://www.allthingsdistributed.com/2007/10/amazons_dynamo.html

Werner Vogels has said "Dynamo is not directly exposed externally as a web service; however, Dynamo and similar Amazon technologies are used to power parts of our Amazon Web Services, such as S3." http://www.allthingsdistributed.com/2007/10/amazons_dynamo.html

我想强调的是，他是不是说S3和迪纳摩共享组件，他明确地说，发电机本身的技术，电源S3之一。我从S3看到的，包括警告，通过假设S3占比一切都是围绕迪纳摩看中了Web服务封装与认证，计费，并在最后写胜冲突解决是用户看不到的。

I would emphasize that he isn't saying S3 and Dynamo share components, he explicitly says that Dynamo itself is one of the technologies that power S3. Everything I've seen from S3, including the caveats, is accounted for by assuming S3 is a fancy web services wrapper around Dynamo with authentication, accounting, and a last-write-wins conflict resolve that is invisible to the user.

原来的问题是关于底层的存储机制S3。这肯定不是像HDFS或类似CouchDB的非关系型数据库的分布式文件系统。发电机填补了这一角色。

The original question was about the underlying storage mechanism for S3. It is explicitly not a distributed file system like HDFS or a non-relational database like CouchDB. Dynamo fills this role.

这篇关于Amazon S3的架构的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！