FS基于数据库而不使用保险丝 [英] FS based on a database without using fuse

问题描述

要从单个目录中提供数百万个文件，能够连接到来自数百个端点的驱动器，以及其他一些原因（为了避免基于gluster / nfs /所有fs的网络解决方案），我想评估基于mongodb（或任何其他）的文件系统的可能性。

基本上，它像保险丝一样工作，每个文件都保存在mongo网格中。理论上，我这样做，

mount mongodbfs / mountPoint mongodb：// localhost

然后当我说 touch /mountPoint/test.txt 这个文件被插入到mongodb中。这个FS也会存储uid / gid和perms文件，我们可以把数百个服务器扔到它上面，而且不需要 useradd 。我不是想包括FS的所有功能，只是我们需要的功能。

我的问题是，我如何开始寻找资源，书籍，链接，人员，开发人员谁会帮我实施这个？至少是一个概念的证明。这可行吗？我应该期待什么作为这样的承诺的时间表？

请只考虑gazillion小文件和文件夹。

ps：经过几天的研究，我认为这是我前进的方向
http://www.ibm.com/developerworks/library/l-sc12.html
http://www.flipcode.com/archives/Programming_a_Virtual_File_System-Part_I.shtml

ps2：我意识到这项工作的难度。然而，我们愿意留下一个严肃的预算，并愿意组建一个严肃的团队来实施它 - 只有在我们确定这不是一个黑洞（因此是一个问题）。

这里最常见的一条建议是使用FUSE。这是极好的建议，你会很好地注意它（如松鼠指出，已经有 GridFS的熔丝这与你想要的非常接近）。

这就是说，如果你想走痛苦和痛苦的漫长道路（编写你自己的文件系统） ，你几乎可以肯定地想要在当地一所大学学习操作系统课程，或者看一些在线课程材料（写一个简单的FS通常是一个小项目，文件系统通常是因为它们是学术玩具）。 Linux文件系统（Moshe Bar）和一些简单的文件系统驱动程序的彻底阅读，看到基本的骨架你需要做的。

就时间轴而言，如果你是一个体面的编码器哟你可以在几天到一周内写一个基本的文件系统（但是它会）。我甚至不会猜测写了一个好的文件系统需要多长时间 - 至少在20世纪70年代末/ 80年代初以来，UFS / FFS（BSD文件系统）一直处于持续发展之中， /增强/错误修复仍然偶尔弹出。 Sun / Oracle的ZFS在其相对较短（6年）的生命周期中已经经历了20次以上的迭代，尽管其中大部分与批量管理能力有关。

To serve millions of files out of a single directory, being able to connect to a drive from hundreds of endpoints, and for some other reasons (to avoid gluster/nfs/all fs based networking solutions), I want to evaluate the possibility of making a filesystem that's based on a mongodb (or any other).

Basically, it works like fusefs, every single file is kept in mongo gridfs. In theory, I do,

mount mongodbfs /mountPoint mongodb://localhost

then when i say touch /mountPoint/test.txt this file is inserted into mongodb. This FS will also store uid/gid and perms with the file, we can throw hundreds of servers to it, and no useradd will be necessary. I'm not thinking to include all the features of FS, just the ones we need.

My question is, how do I start my quest in finding resources, books, links, people, developers who'd help me implement this? at least a proof of concept. Is it feasible? What should I expect as a timeline for such undertaking?

Please only think about gazillion small files and folders.

ps: after a few days of research i think this is the direction i'm heading http://www.ibm.com/developerworks/library/l-sc12.html http://www.flipcode.com/archives/Programming_a_Virtual_File_System-Part_I.shtml

ps2: i'm aware of the difficulty of this undertaking. however we're willing to set aside a serious budget and willing to form a serious team implementing it - only after we make sure that this isn't a black hole (thus the question).

解决方案

Your most frequent piece of advice here is going to be "Use FUSE". This is excellent advice, and you would do well to heed it (As Sciurus pointed out there's already gridfs-fuse which is pretty close to what you want).

That said, if you want to take the long, hard road of pain and suffering (writing your own filesystem), you almost certainly want to take an operating systems course at a local university, or look at some online course materials ("Write a simple FS" is usually a small project. The filesystems typically suck because they're academic toys).
Follow that up with Linux File Systems (Moshe Bar) and a thorough reading of some simple filesystem drivers to see the basic skeleton of what you'll need to do.

As far as timeline, if you're a decent coder you can write a basic filesystem in a few days to a week (but it will SUCK). I wouldn't even guess how long it would take to write a GOOD filesystem -- UFS/FFS (the BSD filesystem) has been under continuous development since at least the late 1970s/early 1980s, and improvements/enhancements/bug fixes still pop up occasionally. Sun/Oracle's ZFS has gone through over 20 iterations in its relative short (6-year) life, though admittedly much of that is related to volume management capabilities.

这篇关于FS基于数据库而不使用保险丝的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！