是否有努力开发具有文件自动更改检测功能的面向构建的文件系统？ [英] Is there an effort to develop build-oriented file system with automatic change detection of files?
问题描述
我最近开始使用Git。
我发现的一个有趣特性是使用哈希来快速检测变化。另一方面，我发现构建工具（如make， ant，javac等）尝试通过检查文件的时间戳来检测源文件中的更改。
这种方法存在的问题是：
- 如果您使用多台
机器，则必须确保所有
时钟同步，否则，需要确保
新文件可能会被认为是
，因为另一台机器的时钟相对于构建机器给出了过去的时间戳。 - 在一个大项目中，您必须扫描所有文件的时间戳以检测变化。
我想知道是否有人已经采用了Git方法来处理这些问题：
- 每个文件都有一个唯一的散列，具体取决于它的内容，而不是时间戳。
- 每个目录也有其散列，具体取决于文件在目录及其哈希中。
- 即使是源码树内部的简单更改，也会由于上述规则而导致根目录具有不同的哈希值
这种机制可以帮助快速构建工具，因为检测源树中的更改只是简单的哈希比较操作。如果源树根目录的散列已经改变，这意味着源树中发生了更深的变化，所以继续对树进行递归扫描以进行更改 - 正如Git检测更改一样。
这并不一定意味着这个源代码树必须由Git管理。
我的想法是文件系统会自动提供文件的哈希代码作为其属性/元数据之一，因此构建工具可以依靠此代替时间戳。此外，目录哈希会自动反映文件的状态。
我已经阅读了一些Sun的ZFS，但我不确定它是否完整解决方案使构建更快。
您如何看待这个想法？
是否已经有这样的文件系统？
是否有这样的构建工具？
我会争辩说，你试图解决的是一个非问题：
使用NTP可以避免时钟偏移问题。
当然，我们可能会同意，在这个问题上抛出一个相当复杂的内容跟踪系统是过分的。
关于性能，扫描整个树在实践中往往不成问题。 stat 的速度非常快（只要你不在Windows上） - ls -lR> / dev / null 在整个Linux内核树（38k个文件）上占用350 ms。
git status 或 git diff ，例如，统计所有文件以检查他们的时间，所以你最好希望这很快。所以如果你想加快 make ，不要看文件系统;相比于任何真正吃掉你的编译时间的东西，它最有可能是微不足道的。 希望能够让你放心！
I recently started to use Git. One of the interesting features I discovered was the use of hashes to quickly detect changes.
On the other hand, I see that build tools (like make, ant, javac, etc.) tries to detect changes in source files by checking file's timestamp.
The problems in this approach are:
- If you work on more than one machine, you have to make sure all clocks are in sync, otherwise, a new file may be considered unchanged because the other machine's clock gave it timestamp of the past relative to the build machine.
- On a big project, you have to scan all files' timestamp in order to detect a change.
I wonder if someone has already taken the Git approach in order to deal with these issues:
- Each file has a unique hash, depending on its content, not timestamp.
- Each directory also has its hash, depending on the files in the directory and their hashes.
- Even the simple change deeply inside the source tree causes the root directory to have a different hash due to the above rules
Such a mechanism could help making build tools much faster, because detecting a change in source tree is a simple operation of hash comparison. If the hash of source tree root directory has changed, it means that a change occurred deeper in the source tree, so continue to scan the tree recursively for changes - exactly as Git does to detect changes.
It doesn't necessarily mean that this source tree has to be managed by Git. My idea is that the file system would automatically provide file's hash code as one of its attributes / metadata, so the build tool could rely on this instead of on timestamp. And in addition, directory hash would automatically reflect the state of the file in it.
I already read a little bit about Sun's ZFS, but I am not sure it's a complete solution to make builds faster.
What do you think about this idea? Is there already such file system? Is there already such build tool?
I'll argue that what you're trying to solve is actually a non-issue:
The clock skew problem can be mostly avoided by using NTP.
Certainly it'd be nice to have clock skew issues eliminated entirely, but we can probably agree that throwing a fairly complex content-tracking system at the problem is overkill.
Regarding performance, scanning the entire tree tends to not be a problem in practice. stat is ridiculously fast (so long as you're not on Windows) -- ls -lR > /dev/null over the entire Linux kernel tree (38k files) takes 350 ms on my system.
In fact, if stat'ing all your files is a problem, then your version control system will become slow, and that will be a much bigger problem than your build performance. Every git status or git diff, for instance, stats all files in your working copy to check their mtimes, so you'd better hope that's fast.
So if you're looking to speed up make, don't look at the file system; it's most likely insignificant compared to whatever is actually eating up your build time.
Hope that eases your mind!
这篇关于是否有努力开发具有文件自动更改检测功能的面向构建的文件系统？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
