并行处理批次项目 [英] Processing batch items in parallel

问题描述

我有一个ItemGroup，并且想要并行处理所有项目(使用自定义任务或.exe).

I have an ItemGroup, and want to process all its items in parallel (using a custom task or an .exe).

• 我可以编写任务/exe来接受整个ItemGroup并在内部并行处理其项目.但是，我希望这种并行性可以与MSBuild的/maxCpuCount参数结合使用，因为否则我可能最终会过度并行化.
• 这个线程说没有办法.
• 我的测试表明，MSBuild的/maxCpuCount仅适用于构建不同的项目，而不适用于项目(请参见下面的代码)

• I could write my task/exe to accept the entire ItemGroup and process its items in parallel internally. However, I want this parallelism to work in conjunction with MSBuild's /maxCpuCount param, since otherwise I might end up over-parallelizing.
• This thread says there's no way.
• My testing shows that MSBuild's /maxCpuCount only works for building different projects, not items (see code below)

如何并行处理ItemGroup中的项目?
有没有办法编写自定义任务以与MSBuild的Parallel支持一起并行工作?

How can I process items from an ItemGroup in parallel?
Is there a way to author a custom task to work in parallel in conjunction with MSBuild's Parallel support?

<Project ToolsVersion="4.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
  <Target Name="Build" >
    <!-- Runs only once - I guess MSBuild detects it's the same project -->
    <!--<MSBuild Projects="$(MSBuildProjectFullPath);$(MSBuildProjectFullPath)" Targets="Wait3000" BuildInParallel="true" />-->

    <!-- Runs in parallel!. Note that b.targets is a copy of the original a.targets -->
    <MSBuild Projects="$(MSBuildProjectFullPath);b.targets" Targets="Wait3000" BuildInParallel="true" />

    <!-- Runs sequentially -->
    <ItemGroup>
      <Waits Include="3000;2000"/>
    </ItemGroup>
    <Wait DurationMs="%(Waits.Identity)" />
  </Target>

  <Target Name="Wait3000">
    <Wait DurationMs="3000" />
  </Target>

  <UsingTask TaskName="Wait" TaskFactory="CodeTaskFactory" AssemblyFile="$(MSBuildToolsPath)\Microsoft.Build.Tasks.v4.0.dll" >
    <ParameterGroup>
      <DurationMs ParameterType="System.Int32" Required="true" />
    </ParameterGroup>
    <Task>
      <Code Type="Fragment" Language="cs">
        Log.LogMessage(string.Format("{0:HH\\:mm\\:ss\\:fff}  Start  DurationMs={1}", DateTime.Now, DurationMs), MessageImportance.High);
        System.Threading.Thread.Sleep(DurationMs);
        Log.LogMessage(string.Format("{0:HH\\:mm\\:ss\\:fff}  End    DurationMs={1}", DateTime.Now, DurationMs), MessageImportance.High);
      </Code>
    </Task>
  </UsingTask>
</Project>   

推荐答案

我知道这很旧，但是如果您有几分钟的时间，请重新尝试使用MSBuild任务.使用Properties和/或AdditionalProperties保留项元数据元素*将解决您在代码示例中描述的问题(仅运行一次-我猜MSBuild检测到它是同一项目").

I know this is old, but if you get a few minutes, revisit your attempt to use the MSBuild task. Using the Properties and/or AdditionalProperties reserved item metadata elements* will resolve the issue you described in your code sample ("Runs only once - I guess MSBuild detects it's the same project").

下面的MSBuild文件通过MSBuild的并行支持(包括/maxCpuCount)并行处理ItemGroup中的项目.它不使用MSBuild Extension Pack中的BuildTargetsInParallel，也不使用任何其他自定义或内联任务.

The MSBuild file below processes items from an ItemGroup in parallel via MSBuild's parallel support (including /maxCpuCount). It does not use BuildTargetsInParallel from the MSBuild Extension Pack, nor any other custom or inline task.

<Project ToolsVersion="4.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">

  <Target Name="Build" >
    <ItemGroup>
      <Waits Include="3000;2000"/>
    </ItemGroup>

    <ItemGroup>
      <ProjectItems Include="$(MSBuildProjectFullPath)">
        <Properties>
          WaitMs=%(Waits.Identity)
        </Properties>
      </ProjectItems>
    </ItemGroup>
    <MSBuild Projects="@(ProjectItems)" Targets="WaitSpecifiedMs" BuildInParallel="true" />
  </Target>

  <Target Name="WaitSpecifiedMs">
    <Wait DurationMs="$(WaitMs)" />
  </Target>

</Project>

*隐藏在 查看全文