预先考虑一个多GB的文件 [英] Prepending to a multi-gigabyte file

问题描述

没有任何方法可以将单个字符添加到一个多字节的文件（在我的实际情况下是一个40GB的文件）限制执行这个。这意味着它可以通过一个工具，一个shell脚本，任何编程语言的程序，...

解决方案

很简单的解决方案有没有系统调用prepend数据，只能追加或重写。

但取决于你正在做的文件，你可能会逃脱技巧。
如果这个文件是按顺序使用的，你可以制作一个命名管道并把 cat onecharfile.txt bigfile> namedpipe 然后使用namedpipe作为文件。同样可以通过 cat onecharfile.txt bigfile |来实现程序如果你的程序需要stdin作为输入。

对于随机访问，FUSE文件系统可以完成，但可能对此太复杂了。 / p>

如果您想让自己的手变得很脏，请指出如何分配

• 一个数据块（关于inode和数据块结构的）


• 插入它作为第二块（或第一块，然后你几乎完成）成为一个文件的链


• 将文件的开头写入该块
  
• 写单个字符作为文件中的第一个
  
• 标记第一个块，就好像它只使用了一个可用的有效负载字节（这可能是为了 last 块，我不知道是否有可能在文件链中间的块）
  
  $ b

  这有可能主要破坏你的文件系统，所以不建议;很好玩。
  

  What would be the most performant way to prepend a single character to a multi-gigabyte file (in my practical case, a 40GB file).

  There is no limitation on the implementation to do this. Meaning it can be through a tool, a shell script, a program in any programming language, ...

  解决方案

  There is no really simple solution. There are no system calls to prepend data, only append or rewrite.

  But depending on what you're doing with the file, you may get away with tricks. If the file is used sequentially, you could make a named pipe and put cat onecharfile.txt bigfile > namedpipe and then use "namedpipe" as file. The same can be achieved by cat onecharfile.txt bigfile | program if your program takes stdin as input.

  For random access a FUSE filesystem could be done, but probably waay too complicated for this.

  If you want to get your hands really dirty, figure out howto

  • allocate a datablock (about inode and datablock structure)
  • insert it into a file's chain as second block (or first and then you're practically done)
  • write the beginning of file into that block
  • write the single character as first in file
  • mark first block as if it uses only one byte of available payload (this is possible for last block, I don't know if it's possible for blocks in middle of file chain).

  This has possibilities to majorly wreck your filesystem though, so not recommended; good fun.

  这篇关于预先考虑一个多GB的文件的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！