可扩展的图像存储 [英] Scalable Image Storage

问题描述

我目前正在为基于 Web 的应用程序设计架构，该架构还应提供某种图像存储.用户将能够上传照片作为该服务的主要功能之一.同时查看这些图像将是主要用途之一(通过网络).

I'm currently designing an architecture for a web-based application that should also provide some kind of image storage. Users will be able to upload photos as one of the key feature of the service. Also viewing these images will be one of the primary usages (via web).

但是，我不确定如何在我的应用程序中实现这样一个可扩展的图像存储组件.我已经想过不同的解决方案，但由于缺少经验，我期待听到您的建议.除了图像，还必须保存元数据.以下是我的初步想法:

However, I'm not sure how to realize such a scalable image storage component in my application. I already thought about different solutions but due to missing experiences, I look forward to hear your suggestions. Aside from the images, also meta data must besaved. Here are my initial thoughts:

1. 使用 HDFS 等(分布式)文件系统，并准备专用网络服务器作为文件系统客户端"，以保存上传的图像和服务请求.图像元数据保存在附加数据库中，包括每个图像的文件路径信息.

1. Use a (distributed) filesystem like HDFS and prepare dedicated webservers as "filesystem clients" in order to save uploaded images and service requests. Image meta data are saved in a additional database including the filepath information for each image.

在 HDFS 之上使用面向 BigTable 的系统，如 HBase，并将图像和元数据保存在一起.同样，网络服务器桥接图像上传和请求.

Use a BigTable-oriented system like HBase on top of HDFS and save images and meta data together. Again, webservers bridge image uploads and requests.

使用像 CouchDB 这样的完全无模式的数据库来存储图像和元数据.此外，通过使用基于 HTTP 的 RESTful API，使用数据库本身进行上传和传送.(附加问题:CouchDB 确实通过 Base64 保存 blob.但是它能否以图像/jpeg 等形式返回数据)?

Use a completly schemaless database like CouchDB for storing both images and metadata. Additionally, use the database itself for upload and delievery by using the HTTP-based RESTful API. (Additional question: CouchDB does save blobs via Base64. Can it however return data in form of image/jpeg etc.)?

推荐答案

为此，我们一直在使用 CouchDB，将图像保存为附件".但一年后，数十 GB 的 CouchDB 数据库文件变得令人头疼.例如，如果您将 CouchDB 复制用于非常大的文档大小，它仍然存在问题.

We have been using CouchDB for that, saving images as an "Attachment". But after a year the multi-dozen GB CouchDB Database files turned out to be a headache. For example CouchDB replication still has issues if you use it with very large document sizes.

所以我们只是重新编写了我们的软件，使用 CouchDB 存储图像信息，使用 Amazon S3 存储实际图像.代码位于 http://github.com/hudora/huImages

So we just rewrote our software to use CouchDB for image information and Amazon S3 for the actual image storage. The code is available at http://github.com/hudora/huImages

您可能希望在现场为您的项目设置与 Amazon S3 兼容的存储服务.这让您保持灵活性，并且暂时无需外部服务即可选择亚马逊.Walruss 似乎成为最受欢迎和可扩展的 S3 克隆.

You might want to set up a Amazon S3 compatible Storage Service on-site for your project. This keeps you flexible and leaves the amazon option without requiring external services for now. Walruss seems to become the most popular and scalable S3 clone.

我还敦促您通过他们出色的开源MogileFS 和 Perlbal 产品.这种组合可能是最著名的图像服务设置.

I also urge you to look into the Design of Livejournal with their excellent Open Source MogileFS and Perlbal offerings. This combination is probably the most Famous image serving setup.

此外，flickr 架构 也可以是一种灵感，尽管它们不向公开，就像 Livejournal 一样.

Also the flickr Architecture can be an inspiration, although they don't offer Open Source software to the public, like Livejournal does.

这篇关于可扩展的图像存储的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！