持久性:数据树存储为目录树 [英] Persistence: Data Trees stored as Directory Trees

问题描述

我想知道将内存树结构存储为目录树以实现持久性的实用性.在我的情况下，他的目标文件系统将是ZFS，一旦创建了结构，它将很少被多个进程访问.

推荐答案

为了读写树，您将在每个节点上多次调用文件系统.这比您可以设计出用于遍历内存映像的任何理智的代码要昂贵得多.

In order to read and write your tree, you will be calling the filesystem several times per node. This is much more expensive than any sane code you could devise to walk a memory image.

这是否明智，取决于您的使用模式.如果在典型的代码调用中您希望读取整个树结构，然后对其进行处理，然后将其完整写出，则最好将其编组为一个文件.但是，如果您希望仅读取/处理/更改少数几个节点，而却没有读取大多数树，则遍历目录结构与进行多次查找/读取之间的性能差异一棵存储在单个文件中的树会小得多，为了简单/清晰/避免重新发明轮子，很可能值得做前者.而且，如果多个进程同时执行此操作，则使用基于目录的方法锁定节点和子树变得更加容易.

Whether it is a sensible approach depends on what your usage pattern is expected to be. If in a typical invocation of your code you expect to read in the entire tree structure, work on it, then write it out in full - you are better off marshalling it up into a single file. If, however, you expect to read/work on/mutate just a few nodes, without reading in most of the tree, the difference in performance between walking the directory structure and doing multiple seeks/reads to traverse a tree stored in a single file will be much smaller, and it may well become worth doing the former for simplicity / clarity / avoiding reinventing wheels. Moreover, if multiple processes are doing this simultaneously, locking nodes and subtrees becomes much easier with the directory-based approach.

请注意，对于某些常用文件系统，打开目录条目的时间取决于目录中条目的总数.

Be aware that for some commonly used filesystems the time to open a directory entry depends on the total number of entries in the directory.

对于网站的CGI后端，我已经用ext3做过类似的事情；没有重新发明轮子，从而使原型设计更快，维护更简单，读/写/锁定扩展得很好，但是非常频繁的更改-每秒数百次-对目录结构本身的更改在实际存储中的效果很差;最后，我对事物进行了重组，以使目录树中经常添加/删除目录条目的部分最终出现在tmpfs卷上-对我而言，这组状态((昂贵地)可以从非易失性较低的存储中重建)重新启动后.我对ZFS的经验很少，并且不知道您打算使用的使用模式，所以不知道这是否会对您造成问题.如果我现在是针对一个使用率非常高的网站执行此操作，那么我可能会改用自己的命名锁库.

I've done similar things with ext3 for a site's CGI backend; not reinventing the wheel made prototyping quicker and maintenance simpler, reads/writes/locking scaled pretty well, but very frequent changes - on the order of hundreds per second - to the directory structure itself worked poorly on real storage; in the end I restructured things so that the sections of directory tree to which directory entries would very frequently be added/removed ended up on a tmpfs volume - for me this set of state could (expensively) be reconstructed from that stored in less volatile storage following a reboot. I have little experience of ZFS, and don't know your intended usage pattern, so don't know if this would be a problem for you. Were I now doing this for a very heavily used site, I would probably roll my own named lock library instead.

这篇关于持久性:数据树存储为目录树的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！