使用.INTERMEDIATE在makefile中建立不可靠的并行构建吗? [英] Unreliable parallel builds in a makefile with .INTERMEDIATE?

问题描述

我有一个生成多个输出文件的工具，众所周知，这很难在make中建模.我在 GNU上使用食谱Makefile规则可从单个源文件生成一些目标，这似乎简单可靠.而且它几乎有效.

I have a tool which generates multiple output files, which is notoriously hard to model in make. I'm using the recipe at GNU Makefile rule generating a few targets from a single source file, which seems simple and reliable. And it almost works.

不幸的是，关于并行构建，我看到了一些非常奇怪的行为，有时它似乎正在删除依赖项.我不明白为什么.

Unfortunately, I'm seeing some very odd behaviour with regard to parallel builds, where it seems to be dropping dependencies sometimes; and I don't understand why.

这是我的测试用例:

out3: out1 out2
    touch out3

.INTERMEDIATE: out.intermediate
out1 out2: out.intermediate
out.intermediate: in
    touch out1 out2

如果我构建一次，它将起作用:

If I build it once, it works:

$ touch in
$ make -f test.mk out3 -j4
touch out1 out2
touch out3

out1和out2一起构建一次，这很好.然后从结果中构建out3.

out1 and out2 are built together, once, which is good; then out3 is built from the result.

现在，我触摸输入文件，模拟增量构建，然后重试:

Now I touch the input file, simulating an incremental build, and try again:

$ touch in
$ make -f test.mk out3 -j4
touch out1 out2

那是正确地重建了out1和out2 ...但是它没有重建它应该具有的out3.然后，如果我进行另一次构建:

That's rebuild out1 and out2, correctly... but it hasn't rebuilt out3, which it should have. Then if I do another build:

$ make -f test.mk out3 -j4
touch out3

...然后赶上了.

仅 适用于并行构建. -j1可以正常工作.

This only applies to parallel builds. -j1 builds work fine.

这很糟糕---我需要能够依靠正确的构建.有谁知道发生了什么事吗?

This is bad --- I need to be able to rely on correct builds. Does anyone have any idea as to what's going on?

这是GNU Make 4.1.

This is GNU Make 4.1.

推荐答案

您链接到的SO答案有错误；令人惊讶的是，如此多的人显然已经成功地使用了它.

The SO answer you link to has an error; surprising that so many people have apparently used it with success.

发生了什么(您可以在make调用中添加-rRd选项以查看更多详细信息)是GNU make的目录缓存的结果. GNU make不会期望文件系统状态除了makefile所描述的以外不会发生任何变化，因此它缓存目录的内容以显着提高性能(对于大型makefile/目录).

What's happening (you can add the -rRd options to your make invocation to see in more detail) is a result of GNU make's directory caching. GNU make doesn't expect the filesystem state to change in any way other than what your makefile describes, and so it caches the contents of directories to provide a significant performance increase (for large makefiles/directories).

基本上，当make运行规则时:

Basically, when make runs the rule:

out.intermediate: in
        touch out1 out2

除了规则中列出的目标:out.intermediate，它预计不会更新任何目标.如果其他文件out1和out2已经在目录高速缓存中进行了内部化(由于它们已经存在并且我们已经将它们作为out3的先决条件进行了检查，所以它们将成为内部文件)，那么make将不会返回到文件系统，查看它们是否已更新:使知道"它们不能更改，因为根据makefile的规定，运行该文件的规则无法对其进行更改.

it doesn't expect that this recipe will update any targets other than the one listed in the rule: out.intermediate. If the other files out1 and out2 are already internalized in the directory cache (which they will be since they exist and we've already checked them as prerequisites to out3), then make won't go back to the filesystem and see if they've been updated or not: make "knows" they can't have changed because no rule that it ran could have changed them, according to the makefile.

有一个简单的单字符修复程序可以使所有这些工作正常进行.更改此行:

There's a simple, one-character fix to make all this work. Change this line:

out1 out2: out.intermediate

对此:

out1 out2: out.intermediate ;

如果您想更加明确，也可以使用以下方法:

If you want to be more explicit you could also use this:

out1 out2: out.intermediate
        @:

或什至用于调试:

out1 out2: out.intermediate
        @echo Do nothing

所有这些的共同点在于，您现在不仅定义了目标和先决条件之间的依赖关系，而且还给出了一个make应当为该规则调用的食谱.即使配方为空(如第一个示例中所示)，因此make实际上并未运行任何命令，make仍将推断out1和/或out2上的时间戳可能已更改，并且它将使这些目标的缓存修改时间无效，然后从文件系统中重新获取它们.

What all these have in common is that you've now defined not just a dependency relationship between the targets and prerequisite, but you've also given a recipe that make should invoke for that rule. Even if the recipe is empty (as in the first example), so make doesn't actually run any commands, make will still infer that it's possible that the timestamps on out1 and/or out2 have changed, and it will invalidate the cached modified time for those targets and re-retrieve them from the filesystem.

这篇关于使用.INTERMEDIATE在makefile中建立不可靠的并行构建吗?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！