VPATH外包源构建 [英] VPATH out-of-source build
问题描述
背景
我们为Matlab和Octave都构建了.mex文件.源代码对于Matlab和Octave都是通用的，但是在这里和那里进行了一些细微的修改(请注意代码中的宏条件).供参考，源结构为:
root_dir/
|
---------------------------------------------------
| | | |
configure.ac mex/ Makefile.am non-mex-source/
|
------------------------------
| |
sources/ build/
| |
----------- -------------------------------
| | | | | |
mex1/ ... mexN/ octave/ mex1.am ... mexN.am matlab/
| | |
----------- ------------- -------------
| | | | | |
file1.cc ... fileM.cc configure.ac Makefile.am configure.ac Makefile.am
根configure.ac和Makefile.am控制mex/build/matlab/，mex/build/octave/中的配置和编译以及我们在non-mex-source/中创建的非mex二进制文件.
.mex源文件位于mex/sources/mex*/中，我们编译了多个.mex文件.
Matlab构建系统位于mex/build/matlab/中.根目录中的configure.ac和Makefile.am通过AC_CONFIG_SUBDIRS([mex/build/matlab])从根目录中的configure.ac调用. /mex/build/octave/的设置相同，其中configure.ac已被修改以查找特定于Octave的标头和库.
Matlab和Octave共有的automake构建指令(源文件的路径，要构建的.mex文件的路径)存在于名为/mex/build/mex*.am的文件中，并且通常如下所示:
mex_PROGRAMS += mexI
nodist_mexI_SOURCES = \
../../sources/mexI/mexI.c \
../../sources/mexI/mexI_file1.c \
../../sources/mexI/mexI_fileN.c
递归make非常适合我们的设置(我们曾经有名为/mex/build/matlab/mex*/和/mex/build/octave/mex*/的目录包含Makefile.am文件；目标文件将在此处编译，而输出文件将在此处). /p>
问题
鉴于GNU subdir-objects所做的更改，我们正在转向非递归版本.我已经想出了一种解决方案，其中将源软链接到/mex/build/matlab/mex*/和/mex/build/octave/mex*/目录中(使用BUILT_SOURCES = .mexfileslinked，并设置源以取决于此文件的存在；该规则第一次编译源代码时，此文件将链接到源代码和touch .mexfileslinked).效果很好，但我认为必须有一个更优雅的解决方案.
我创建了一个最小的工作示例，该示例显示了当前设置以及运行autoreconf -si时遇到的警告.文件如下:
root_dir/configure.ac:
AC_PREREQ([2.69])
AC_INIT([soquestion], [2.0])
AC_CONFIG_SRCDIR([configure.ac])
AM_INIT_AUTOMAKE([1.15.1 foreign])
# config stuff for other, non-mex binaries we are building
AC_CONFIG_SUBDIRS([mex/build/matlab])
AC_CONFIG_SUBDIRS([mex/build/octave])
AC_CONFIG_FILES([Makefile])
AC_OUTPUT
root_dir/Makefile.am:
SUBDIRS = mex/build/matlab mex/build/octave
root_dir/mex/build/mex1.am:
mex_PROGRAMS = mex1
mexdir = $(libdir)/mex/matlab
nodist_mex1_SOURCES = \
$(top_srcdir)/../../sources/mex1/mex1.cc
root_dir/mex/build/matlab/configure.ac和
root_dir/mex/build/octave/configure.ac:
AC_PREREQ([2.69])
AC_INIT([soquestion], [2.0])
AC_CONFIG_SRCDIR([configure.ac])
AM_INIT_AUTOMAKE([1.15.1 foreign])
AC_PROG_CXX
AC_CONFIG_FILES([Makefile])
AC_OUTPUT
root_dir/mex/build/matlab/Makefile.am和root_dir/mex/build/octave/Makefile.am:
include ../mex1.am
从root_dir运行autoreconf -si会导致警告:
../mex1.am:5: warning: source file '$(top_srcdir)/../../sources/mex1/mex1.cc' is in a subdirectory,
../mex1.am:5: but option 'subdir-objects' is disabled
Makefile.am:1: '../mex1.am' included from here
automake: warning: possible forward-incompatibility.
automake: At least a source file is in a subdirectory, but the 'subdir-objects'
automake: automake option hasn't been enabled. For now, the corresponding output
automake: object file(s) will be placed in the top-level directory. However,
automake: this behaviour will change in future Automake versions: they will
automake: unconditionally cause object files to be placed in the same subdirectory
automake: of the corresponding sources.
automake: You are advised to start using 'subdir-objects' option throughout your
automake: project, to avoid future incompatibilities.
../mex1.am:5: warning: source file '$(top_srcdir)/../../sources/mex1/mex1.cc' is in a subdirectory,
../mex1.am:5: but option 'subdir-objects' is disabled
Makefile.am:1: '../mex1.am' included from here
automake: warning: possible forward-incompatibility.
automake: At least a source file is in a subdirectory, but the 'subdir-objects'
automake: automake option hasn't been enabled. For now, the corresponding output
automake: object file(s) will be placed in the top-level directory. However,
automake: this behaviour will change in future Automake versions: they will
automake: unconditionally cause object files to be placed in the same subdirectory
automake: of the corresponding sources.
automake: You are advised to start using 'subdir-objects' option throughout your
automake: project, to avoid future incompatibilities.
当然，我可以简单地将subdir-objects添加到AM_INIT_AUTOMAKE命令中以删除警告.但是，然后我会遇到这里，则由八度构建认为由matlab构建创建的目标文件是有效的，从而导致八度构建失败(八度构建在matlab构建之后运行).
为解决该问题，我想从mex/build/matlab/objects构建Matlab .mex文件，从mex/build/octave/objects构建Octave .mex文件(或者更好的是，建立mex/build/matlab/objects/mex1 ... mex/build/matlab/objects/mexN ).我知道我们通常可以切换到这些目录，然后运行../configure，但由于AC_CONFIG_SUBDIRS仅将子目录作为参数，因此我看不到如何从root_dir/configure.ac进行操作.
我已经想出了一种解决方案，其中源被软链接到
/mex/build/matlab/mex*/和/mex/build/octave/mex*/目录[...]，但是我离开时必须考虑必须有一个更优雅的解决方案.
实际上，正如我在回答上一个问题时所写的那样，最优雅的解决方案通常是使用源代码之外的版本，Autotools会自动为您提供这些版本.至少还有另一种选择，我将在后面介绍.
您继续说
从root_dir运行autoreconf -si会导致[关于不使用Automake的
subdir-objects选项的警告....]当然，我可以简单地将子目录对象添加到AM_INIT_AUTOMAKE命令中以删除警告.但是，然后我会遇到[在上一个问题中]概述的问题.
通常，外包源构建仍然是解决此类问题的最佳解决方案.您可以打开subdir-objects进行源代码外构建以使警告消失.当它们不同时，构建的文件仍将进入构建树，而不是源树.启用该选项后，它们将以与源树相同的目录结构排列在构建树中，这是一件好事和有用的事情.因此，如果对每种目标类型都进行单独的源外构建，则将有单独的输出.
要解决该问题，我想从
mex/build/matlab/objects构建Matlab.mex文件，从mex/build/octave/objects构建Octave.mex文件(或者更好的是，建立mex/build/matlab/objects/mex1...mex/build/matlab/objects/mexN).
您似乎在关注特定的解决方案思路，而不是更广泛地考虑整个问题.您可以使您的中间对象文件构建在您选择的目录中.为此，您需要创建显式的make规则以在所需的目标目录中构建它们，并创建其他显式的规则以从这些最终目录中构建最终输出.到那时，您将扔掉Automake提供的许多优点，而大多数情况下，它们只是直接编写普通的Makefile.如果愿意，您可以这样做，但是我认为您将付出不必要的努力，并且生成比以前可能更脆弱的构建系统.
在这方面，您需要了解，就make而言，以make表示的目标文件所在目录的名称是该目标标识符的组成部分.您不能仅通过拨动开关在另一个目录中构建相同文件-如果您在另一个目录中构建某个文件，则出于这个原因，它就是一个不同的文件.然后，所有需要将该文件作为目标或依赖项的make规则都需要更改.
我知道我们通常可以切换到这些目录，然后运行../configure，但我看不到如何从root_dir/configure.ac执行此操作，因为AC_CONFIG_SUBDIRS仅将子目录作为参数.
好吧，根目录configure.ac是您在上一个问题中没有提到的新皱纹.通过使用AC_CONFIG_SUBDIRS将mex组件关联为较大项目的子项目，至少在主项目的一个版本的上下文中，您将失去针对这两个项目执行单独的源外版本的能力. .此外，因此，正如您先前的问题所断言的那样，您不生成的是非递归版本–您只是稍微放宽了版本.至少，顶层Makefile仍将递归到子项目Makefile中.
但是，如果顶层项目对子项目的构建输出没有构建依赖关系，那就很好.在那种情况下，相对于主项目而言，非递归构建相对于递归构建的优势基本上都不会应用到那些文件.这就为子项目使用源代码外的构建打开了大门，但是您将需要使用不同于AC_CONFIG_SUBDIRS的机制来配置它们.
我建议使用AC_CONFIG_COMMANDS来设置单独的源外版本.这不会像AC_CONFIG_SUBDIRS那样干净，它放弃了递归帮助，但是它提供了所需的灵活性.也许遵循以下思路(未经测试):
AC_CONFIG_COMMANDS([mex-matlab], [
mkdir -p mex-matlab
cd mex-matlab
"$[]top_srcdir"/mex/build/matlab/configure
], [top_srcdir=`dirname "$[]0"`])
AC_CONFIG_COMMANDS([mex-octave], [
mkdir -p mex-octave
cd mex-octave
"$[]top_srcdir"/mex/build/octave/configure
], [top_srcdir=`dirname "$[]0"`])
然后，顶层Makefile.am需要在其SUBDIRS变量中指定子项目的构建目录(在上面的示例中为mex-matlab和mex-octave).
至少还有另一种完全不同的方法:Automake内置支持使用不同的选项从同一来源构建不同的目标.您不会在当前的构建系统中利用它，因为您已将octave和matlab的位分离到了单独的构建中，但是如果将它们合并到一个基于Automake的构建系统中(可能有条件地构建了这两个部分)，则Automake应该避免目标文件冲突在需要的地方.它通过为中间文件使用不同的名称来实现此目的，而不是将它们放在不同的目录中.因此，也可以通过这种方法添加subdir-objects选项来解决Automake的警告.
关键部分可能遵循以下一般模式:
文件(请注意:) mex/build/Makefile.am:
mex_PROGRAMS += matlab/mexI octave/mexI
nodist_matlab_mexI_SOURCES = \
../sources/mexI/mexI.c \
../sources/mexI/mexI_file1.c \
../sources/mexI/mexI_fileN.c
# Need at least one target-specific flags variable, but that will
# come naturally in your case
matlab_mexI_CPPFLAGS = ...
nodist_octave_mexI_SOURCES = \
../sources/mexI/mexI.c \
../sources/mexI/mexI_file1.c \
../sources/mexI/mexI_fileN.c
# ...
当然，由于使用这种方法，您不需要为mex文件使用单独的构建系统，因此您可以考虑完全或部分展平mex/build子树.
我之所以提到第二个替代方案，是因为它可能需要对您现有的构建系统进行比其他构建系统更大的修改，并且还因为我在回答您的其他问题时已经谈到了源代码外的构建.但是，对于当前问题，您正在将mex位设置为主项目的子项目的复杂性稍微改变了微积分.从Autotools的角度来看，此替代方法比为子项目安排单独的外包源构建更干净.
Background
We build .mex files for both Matlab and Octave. The source code is common to both Matlab and Octave but with slight modifications here and there (taken care of via macro conditionals in the code). For reference, the source structure is:
root_dir/
|
---------------------------------------------------
| | | |
configure.ac mex/ Makefile.am non-mex-source/
|
------------------------------
| |
sources/ build/
| |
----------- -------------------------------
| | | | | |
mex1/ ... mexN/ octave/ mex1.am ... mexN.am matlab/
| | |
----------- ------------- -------------
| | | | | |
file1.cc ... fileM.cc configure.ac Makefile.am configure.ac Makefile.am
The root configure.ac and Makefile.am control the configuration and compilation in mex/build/matlab/, mex/build/octave/ and the non-mex binaries we create in non-mex-source/.
The .mex source files are located in mex/sources/mex*/ and we compile multiple .mex files.
The Matlab build system is located in mex/build/matlab/. The configure.ac and Makefile.am located here are called from the configure.ac in the root directory via AC_CONFIG_SUBDIRS([mex/build/matlab]). The same setup holds true for /mex/build/octave/, where the configure.ac has been modified to find Octave-specific headers and libraries.
The automake build instructions (the paths to the source files, the .mex files to be built) common to both Matlab and Octave exist in the files named /mex/build/mex*.am and typically look like:
mex_PROGRAMS += mexI
nodist_mexI_SOURCES = \
../../sources/mexI/mexI.c \
../../sources/mexI/mexI_file1.c \
../../sources/mexI/mexI_fileN.c
Recursive make worked perfectly for our setup (we used to have directories named /mex/build/matlab/mex*/ and /mex/build/octave/mex*/ that contained Makefile.am files; object files would be compiled there and the output files would be located there).
Problem
We are moving to a non-recursive build given the changes that GNU subdir-objects change. I have already come up with a solution where the sources are soft-linked into the /mex/build/matlab/mex*/ and /mex/build/octave/mex*/ directories (using BUILT_SOURCES = .mexfileslinked and seting up the sources to depend on the existence of this file; the rule for this file being to link in the sources and touch .mexfileslinked) when the source is compiled the first time through. This works well but I am left thinking that there must be a more elegant solution.
I have created a minimum working example that shows the current setup and the warnings encountered when we run autoreconf -si. The files are as follow:
root_dir/configure.ac:
AC_PREREQ([2.69])
AC_INIT([soquestion], [2.0])
AC_CONFIG_SRCDIR([configure.ac])
AM_INIT_AUTOMAKE([1.15.1 foreign])
# config stuff for other, non-mex binaries we are building
AC_CONFIG_SUBDIRS([mex/build/matlab])
AC_CONFIG_SUBDIRS([mex/build/octave])
AC_CONFIG_FILES([Makefile])
AC_OUTPUT
root_dir/Makefile.am:
SUBDIRS = mex/build/matlab mex/build/octave
root_dir/mex/build/mex1.am:
mex_PROGRAMS = mex1
mexdir = $(libdir)/mex/matlab
nodist_mex1_SOURCES = \
$(top_srcdir)/../../sources/mex1/mex1.cc
root_dir/mex/build/matlab/configure.ac and
root_dir/mex/build/octave/configure.ac:
AC_PREREQ([2.69])
AC_INIT([soquestion], [2.0])
AC_CONFIG_SRCDIR([configure.ac])
AM_INIT_AUTOMAKE([1.15.1 foreign])
AC_PROG_CXX
AC_CONFIG_FILES([Makefile])
AC_OUTPUT
root_dir/mex/build/matlab/Makefile.am and root_dir/mex/build/octave/Makefile.am:
include ../mex1.am
Running autoreconf -si from root_dir results in the warnings:
../mex1.am:5: warning: source file '$(top_srcdir)/../../sources/mex1/mex1.cc' is in a subdirectory,
../mex1.am:5: but option 'subdir-objects' is disabled
Makefile.am:1: '../mex1.am' included from here
automake: warning: possible forward-incompatibility.
automake: At least a source file is in a subdirectory, but the 'subdir-objects'
automake: automake option hasn't been enabled. For now, the corresponding output
automake: object file(s) will be placed in the top-level directory. However,
automake: this behaviour will change in future Automake versions: they will
automake: unconditionally cause object files to be placed in the same subdirectory
automake: of the corresponding sources.
automake: You are advised to start using 'subdir-objects' option throughout your
automake: project, to avoid future incompatibilities.
../mex1.am:5: warning: source file '$(top_srcdir)/../../sources/mex1/mex1.cc' is in a subdirectory,
../mex1.am:5: but option 'subdir-objects' is disabled
Makefile.am:1: '../mex1.am' included from here
automake: warning: possible forward-incompatibility.
automake: At least a source file is in a subdirectory, but the 'subdir-objects'
automake: automake option hasn't been enabled. For now, the corresponding output
automake: object file(s) will be placed in the top-level directory. However,
automake: this behaviour will change in future Automake versions: they will
automake: unconditionally cause object files to be placed in the same subdirectory
automake: of the corresponding sources.
automake: You are advised to start using 'subdir-objects' option throughout your
automake: project, to avoid future incompatibilities.
Of course, I could simply add subdir-objects to the AM_INIT_AUTOMAKE command to remove the warnings. But, then I would run into the problem outlined here, where the object files created by the matlab build are viewed as valid by the octave build, causing the octave build to fail (the octave build being run after the matlab build).
To solve that problem, I'd like to build the Matlab .mex files from mex/build/matlab/objects and the Octave .mex files from mex/build/octave/objects (or, even better, mex/build/matlab/objects/mex1 ... mex/build/matlab/objects/mexN). I understand that we could usually change into these directories then run ../configure but I don't see how to do this from the root_dir/configure.ac as AC_CONFIG_SUBDIRS takes only the sub directory as an argument.
I have already come up with a solution where the sources are soft-linked into the
/mex/build/matlab/mex*/and/mex/build/octave/mex*/directories [...] but I am left thinking that there must be a more elegant solution.
Indeed, as I wrote in answer to your previous question, the most elegant solution would normally be to use out-of-source builds, which the Autotools make available to you automatically. There is at least one other alternative, too, which I'll describe later.
You go on to say,
Running autoreconf -si from root_dir results in [warnings about not using Automake's
subdir-objectsoption ....] Of course, I could simply add subdir-objects to the AM_INIT_AUTOMAKE command to remove the warnings. But, then I would run into the problem outlined [in my previous question].
Out-of-source building is still usually the best solution to problems of this sort. You can turn on subdir-objects for your out-of-source builds to silence the warning; the built files will still go into the build tree, not the source tree, when those differ. With the option enabled, they will be arranged in the build tree in a directory structure parallel to that of the source tree, which is a good and useful thing. Thus, if you do a separate out-of-source build for each target type then you will have separate outputs.
To solve that problem, I'd like to build the Matlab
.mexfiles frommex/build/matlab/objectsand the Octave.mexfiles frommex/build/octave/objects(or, even better,mex/build/matlab/objects/mex1...mex/build/matlab/objects/mexN).
You seem to be focusing on a particular solution idea instead of considering the overall problem more broadly. You can cause your intermediate object files to be built in the directory of your choice. To do so, you would need to create explicit make rules for building them in their desired target directories, and additional explicit rules for building the final outputs from those. At that point, you would be throwing away many of the advantages that Automake provides, and mostly just writing ordinary Makefiles, directly. You can do that if you want to do, but I think you would be spending more effort than necessary, and producing a more fragile build system than you might otherwise have done.
In this regard, you need to understand that the name of the directory in which a target file is built, as expressed to make, is an integral part of that target's identifier as far as make is concerned. You cannot just build the same file in a different directory by flipping a switch -- if you build something in a different directory then for that reason, it is a different file. All the make rules involving that file as either a target or a dependency then need to change.
I understand that we could usually change into these directories then run ../configure but I don't see how to do this from the root_dir/configure.ac as AC_CONFIG_SUBDIRS takes only the sub directory as an argument.
Well, the root-directory configure.ac is a new wrinkle that you didn't mention in your previous question. By using AC_CONFIG_SUBDIRS to associate the mex components as sub-projects of a larger project, you lose the ability to perform separate out-of-source builds for those two, at least in the context of one build of the master project. Additionally, you are therefore not producing a non-recursive build, as your earlier question asserted -- you are just flattening out your build a bit. At minimum, the top-level Makefile will still recurse into the subproject Makefiles.
If the top-level project has no build dependencies on the subprojects' build outputs, however, then that's just fine. In that case, substantially none of the advantages that non-recursive builds have over recursive ones would apply to those files, relative to the main projects'. That holds open the door to using out-of-source builds for the sub-projects, but you will need to use a different mechanism than AC_CONFIG_SUBDIRS to configure them.
I would suggest using AC_CONFIG_COMMANDS to set up the separate out-of-source builds. This will not be quite as clean as AC_CONFIG_SUBDIRS would be, and it gives up recursive help, but it provides the needed flexibility. Maybe something along these lines (untested):
AC_CONFIG_COMMANDS([mex-matlab], [
mkdir -p mex-matlab
cd mex-matlab
"$[]top_srcdir"/mex/build/matlab/configure
], [top_srcdir=`dirname "$[]0"`])
AC_CONFIG_COMMANDS([mex-octave], [
mkdir -p mex-octave
cd mex-octave
"$[]top_srcdir"/mex/build/octave/configure
], [top_srcdir=`dirname "$[]0"`])
The top-level Makefile.am would then need to specify the sub-project build directories (mex-matlab and mex-octave in the above example) in its SUBDIRS variable.
There is also at least one other completely different approach: Automake has built-in support for building different targets from the same sources with different options. You do not leverage that in your current build system because you have segregated the octave and matlab bits into separate builds, but if you merged those into one Automake-based build system that built both pieces (perhaps conditionally) then Automake should take care of avoiding the object-file collisions where needed. It does this by using different names for the intermediate files, instead of putting them in different directories. Thus, with this approach, too, you could just add the subdir-objects option to resolve Automake's warning.
The key parts might follow this general pattern:
File (note well:) mex/build/Makefile.am:
mex_PROGRAMS += matlab/mexI octave/mexI
nodist_matlab_mexI_SOURCES = \
../sources/mexI/mexI.c \
../sources/mexI/mexI_file1.c \
../sources/mexI/mexI_fileN.c
# Need at least one target-specific flags variable, but that will
# come naturally in your case
matlab_mexI_CPPFLAGS = ...
nodist_octave_mexI_SOURCES = \
../sources/mexI/mexI.c \
../sources/mexI/mexI_file1.c \
../sources/mexI/mexI_fileN.c
# ...
Of course, since with this approach you don't want separate build systems for your mex files, you might consider completely or partially flattening the mex/build subtree.
I mention this alternative second because it probably would require much more significant modifications to your existing build system than the other would, and also because I had already talked about out-of-source builds in my answer to your other question. For the present question, however, the complication that you are currently setting up the mex bits as subprojects of the main project alters the calculus a bit. This alternative is cleaner from an Autotools perspective than is arranging separate out-of-source builds for the sub-projects.
这篇关于VPATH外包源构建的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
