您可以在项目列表上进行并行增量构建吗? [英] Can you do parallel incremental building on a list of projects?

问题描述

是否可以列出项目列表并仅在它们不是最新时并行构建它们?

Is it possible to take a list of projects and parallel build them only if they aren't up to date?

<Target Name="DependenciesLevel7" DependsOnTargets="DependenciesLevel6">
<Message Text="5 items to build" />
<MSBuild Projects="C:\Projects\ApplicationManager.csproj;C:\Projects\Metrics.csproj" Properties="$(CustomAllProperties)" BuildInParallel="true">
  <Output TaskParameter="TargetOutputs" ItemName="built_DependenciesLevel7" />
</MSBuild>

这是我正在构建的格式的一个示例，我希望能够仅并行构建此处不是最新的项目?也许内部msbuild任务调用是自动并行的?如果是这样，我将如何设置它，使其在之前的构建任务的基础上递增? (Target DependenciesLevel6)我相信要进行增量构建，您必须在目标中使用输入/输出.

This is an example of the format i'm building in, I was hoping to be able to parallel build only items that aren't up to date here? Perhaps internal msbuild task calls are automatically parallel? If so how would I set this up so that it's incremental based on the previous build task? (Target DependenciesLevel6) I believed for incremental building you have to use Inputs/Outputs in your target.

问题摘要:

• 传递给msbuild任务的项目列表是否自动递增(如果构建已经是最新的，则跳过构建)?
  • 如果没有，是否可以在并行化时对项目列表进行增量处理?
  • Is the list of projects passed to an msbuild task automatically incremental (building is skipped if the build is already up to date)?
    • If not, is it possible to do incremental on a list of projects while parallelizing?

    推荐答案

    您正在描述的并行增量构建"已经在(种类)中构建，但是您真正需要的是并行部分构建

    what you are describing "parallel incremental building" is already built in (kind of) but what you really need is parallel partially building.

    首先让我解释一下这些概念，然后再具体介绍它在您的方案中的工作方式.您需要了解两件事.增量建筑和部分建筑.我刚刚在 http://sedodream.com/2010/09/上写了一篇博客文章对此进行了讨论. 23/MSBuildYouveHeardOfIncrementalBuildingButHaveYouHeardOfPartialBuilding.aspx ，但我将在此处粘贴相关部分.

    Let me first explain the concepts and then I'll circle back how it works in your scenario specifically. There are two things you need to know about; incremental building and partial building. I just wrote a blog post discussing this at http://sedodream.com/2010/09/23/MSBuildYouveHeardOfIncrementalBuildingButHaveYouHeardOfPartialBuilding.aspx but I'll paste the relevant parts here.

    增量构建是仅应构建过时的概念.为了支持此MSBuild，在Target元素上具有属性，输入和输出.使用这些属性，您可以指定进入目标的文件(通过输入属性)，以及期望从目标出来的文件(通过输出属性).完成此操作后，MSBuild会将输入的时间戳记与输出进行比较，如果所有输出都是最新的(即输入较旧)，则将跳过目标.看看下面非常简单的项目文件.

    Incremental building is the concept that you should only build what is out of date. To support this MSBuild has the attributes, inputs and outputs on the Target element. With these attributes you can specify the files that go into a target (via inputs attribute), and the files that you are expecting to come out of a target (via outputs attribute). Once you do this MSBuild will compare the timestamp of the inputs to the outputs and if all outputs are up-to-date (i.e. the inputs are older) then the target will be skipped. Take a look at the very simple project file below.

    <Project xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
    
      <ItemGroup>
        <Files Include="src\01.txt;src\02.txt;src\03.txt;src\04.txt;src\05.txt;"/>
      </ItemGroup>
    
      <PropertyGroup>
        <Dest>dest\</Dest>
      </PropertyGroup>
    
      <Target Name="CopyFiles" 
              Inputs="@(Files)" 
              Outputs="@(Files->'$(Dest)%(Filename)%(Extension)')">
    
        <Message Text="CopyFiles" />
        <Copy SourceFiles="@(Files)"
              DestinationFiles="@(Files->'$(Dest)%(Filename)%(Extension)')"/>
      </Target>
    
      <Target Name="DeleteTwoFiles">
        <Message Text="DeleteTwoFiles" />
        <Delete Files="$(dest)01.txt;$(dest)02.txt"/>
      </Target>
    </Project>
    

    在此项目文件中，我们有两个目标: CopyFiles和DeleteTwoFiles.现在忽略DeleteTwoFiles.另请注意，我正在执行此构建的目录中有一个文件夹src，其中文件"项中列出了文件.在CopyFiles目标上，我指定了输入和输出.输入只是@(Files)，这是目标所作用的文件.输出包含表达式@(Files->'$(Dest)％(Filename)％(Extension)').这与Copy语句中的表达式相同.如果目标文件夹为空，并且我执行了CopyFiles目标，则结果显示如下.

    In this project file we have two targets; CopyFiles and DeleteTwoFiles. Ignore DeleteTwoFiles for now. Also take a note that the directory where I’m executing this build has a folder, src, with the files listed in the Files item. On the CopyFiles target I have specified the inputs and outputs. The inputs is just @(Files), this are the files that the target is acting upon. The outputs contains the expression @(Files->'$(Dest)%(Filename)%(Extension)'). Which is the same expression from the Copy statement. If the Dest folder is empty and I execute the CopyFiles target the result is shown below.

    所以正如预期的那样，文件被复制了，所以一切都很好.现在，如果我再次执行它会怎样?输出如下所示

    So just as expected the files were copied over, so its all good. Now what happens if I execute it again? The output is shown below

    因此，您可以看到目标已被跳过，因此未执行消息语句"CopyFiles"，因此未执行复制操作.简而言之，这就是增量构建. 现在，有了包含所有文件的dest文件夹，您认为我执行命令msbuild.exe PartialBuilding01.proj/t:DeleteTwoFiles; CopyFiles，会发生什么?此命令将首先从输出目录中删除两个文件，然后再次调用CopyFiles目标.让我们看看下面的结果.

    So as you can see the target was skipped, the message statement "CopyFiles" was not executed nor was the copy as a result. So this, in a nutshell, is incremental building. Now, with the dest folder containing all files, what do you think would happen I execute the command msbuild.exe PartialBuilding01.proj /t:DeleteTwoFiles;CopyFiles? This command will first delete two files from the output directory and then call the CopyFiles target again. Let’s see the result below.

    执行CopyFiles目标时，您会看到该语句正在部分构建目标‘CopyFiles’，...".当执行目标MSBuild的时间到了时，检查了输入和输出，确定文件01.txt& 02.txt已过期(因为它们在目标中不存在)，但是03.txt，04.txt和05.txt是最新的.因此，MSBuild将CopyFiles目标提供给文件"项的一个值，该值仅包含01.txt和02.txt，然后让它完成任务.

    When the CopyFiles target was executed you see that statement "Building target ‘CopyFiles’ partially, …". When the time came to execute the target MSBuild examined the inputs and outputs, it determined that the files 01.txt & 02.txt were out of date (because they didn’t exist in the target) but 03.txt, 04.txt and 05.txt were up to date. So MSBuild feed the CopyFiles target a value for the Files item that only contained the 01.txt and 02.txt and let it do its thing.

    现在，这在许多方面与您的问题有关，有些并不像您希望的那样直接.首先，MSBuild将逐步构建您的项目，因此，如果您的项目是最新的，则不会再次构建.事实是，为了使MSBuild确定您的项目是最新的，它必须加载该项目并运行默认目标(通常是Build)，然后目标本身会发现没有工作要做.这些东西本身需要时间.因此，如果您有大量项目，或者项目中包含大量文件，那么您可以自己处理问题.您需要的一种方法是确定项目是否最新，并在输入和输出属性中正确表示该项目.完成此操作后，您应该可以跳过构建最新的项目.

    Now this relates to your problem in many ways some not as direct as you might hope. Firstly MSBuild will incrementally build your project, so if your project is up to date then it will not be built again. The thing is though that in order for MSBuild to determine that your project is up to date it has to load the project run the default target (usually Build) and then the targets themselves will figure out that there is no work to do. This stuff itself takes time. So if you have a huge number of projects, or a huge number of files inside of a project then you can take matters into your own hands. What you need is a way to determine if your projects are up to date or not and correctly express that inside of your inputs and outputs attributes. Once you do this you should be able to skip building the projects which are up to date.

    问题的核心是如何使输入/输出正确.如果您想办法做到这一点，那么您将得到想要的东西.具体操作方式取决于您的情况，但是我可以看到类似这样的东西:

    The core of the problem is how do you craft the inputs/outputs to be correct. If you can think of a way to do that then you will get what you want. How you craft this will depend on your scenario but I could see something like this:

    1. 在每个项目构建完成后，将文件拖放到该项目特定的已知位置
    2. 在构建项目之前，请扫描其目录，找到最新文件，然后将项目文件的时间戳更新为该值
    3. 然后，您可以将项目文件作为输入"值，并将标记文件作为输出"
    4. 然后致电您的目标

    在这种情况下，您假定所有依赖项都完全包含在项目目录下的文件中(可能不正确).我并不是说这是理想的解决方案，而是解决方案.

    In this case you assume that all dependencies are fully contained in files under the directory of the project (which may not be true). I'm not saying this is the ideal solution, but a solution.

    =============================================

    ==============================================

    根据下面的问题进行更新.

    您将需要将项目放入ProjectFiles之类的项目(尽管不是必需的)，然后使用@(ProjectFiles)作为输入.对于输出，我所说的是最困难的部分.您必须找出一种方法来知道(或通过您自己的过程向您表明)项目是最新的.没有为此内置任何内容. 关注增量构建与干净构建.在理想的世界中，增量&干净的版本是相同的.但是有时候情况并非如此.对于许多项目而言.如果您开始向构建过程添加一堆目标，并且将它们设置为进行增量构建，但是没有正确实现，则MSBuild可能会在目标确实过时时跳过目标.一个很好的例子是，当您创建一个目标时，将输入"设置为文件列表，然后将输出"设置为已创建文件的列表.如果您不扩展清理过程以删除那些已创建的文件，则下次重新构建(假设您没有更改文件)时，如果目标应在先前的重新构建中被清理，则将跳过该目标.

    You will want to put the projects into an item (though not required) like ProjectFiles and then use @(ProjectFiles) for inputs. For outputs that is what I was saying is the hard part. You have to figure out a way to know (or indicate to you via your own process) that the projects are up to date. There is nothing built in for this. Concern fo incremental build vs. clean build. In a perfect world incremental & clean builds are the same. But sometimes that is not the case. For many projects it is. If you start adding a bunch of targets to your build process and you set them up to do incremental build, but you do not implement that properly then you may MSBuild may skip targets when they were indeed out of date. A good example of this would be when you create a target with Inputs set to a list of files and then the Outputs set to a list of created files. If you do not extend the clean process to delete those created files, then the next time you Rebuild (assuming you didn't change the files) the target will be skipped when it should have been cleaned on the previous Rebuild.

    这篇关于您可以在项目列表上进行并行增量构建吗?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！