如何运行一个Perl的测试套件并行一些但不是所有的测试？ [英] How to run some but not all tests in a Perl test suite in parallel?

问题描述

我有一个基于Perl的测试套件10000测试，我想使运行速度更快。我已经使用 -j 标志测试，以证明，我发现大多数-但并非所有的我的测试准备并行运行。

虽然我可以使得余下的测试工作是平行友好，我希望总是存在一些测试，哪些不是。什么是管理这个的好办法？我想它很容易高效运行整套测试，并可以很容易地标记测试为不平行就绪如果我需要。

这里有一些选择我看到：

1. 证明可以修补，以支持一些测试不平行就绪


2. 詹金斯被用于管理测试套件运行。我可以在非平行测试分裂成自己的运行。换句话说，放弃使用两个测试运行。


3. 或许有合并的方式两个 TAP 结果流在一起，我还没有恢复过来。

我不是太关心我怎么会管理的例外列表中。要么我可以在一个文件中保持一个列表作为测试工具的基础设施的一部分，或者我可以把在每个测试头的东西，将标志着它是这样，我们的测试工具可以动态地确定例外列表中。

（测试套件是部分基于测试::类，我会还可以看的Test ::类::负载来加快它为好。）

解决方案

我找到了解决办法。它是 aggregate_tests（）为TAP ::线束的文件中。它包括我怎么能写我自己为此线束A code样品：<​​/ P>

  

...这是有用的，例如，在一些测试应的情况下
  并行运行，但其他人不适合并行执行。

 我的$ =格式化TAP ::格式化::控制台 - ＆gt;新建;
我的$ ser_harness = TAP :: Harness-＆gt;新建（{格式化=＆GT; $格式化}）;
我的$ par_harness = TAP :: Harness-＆gt;新建（
    {格式化=＆GT; $格式，
        工作= GT; 9
    }
）;
我的$聚合= TAP ::分析器:: Aggregator-＆gt;新建;$ aggregator-＆GT;启动（）;
$ ser_harness-＆GT; aggregate_tests（$聚合，@ser_tests）;
$ par_harness-＆GT; aggregate_tests（$聚合，@par_tests）;
$ aggregator-＆GT;停止（）;
$ formatter-＆GT;摘要（$聚合器）;
 

从那里，它看起来像我可以：

• 子类应用::证明并重写 _runtests（），这是在上面的新功能可以在被合并。


• 叉证明，使其调用 ::我::应用证明而不是应用::证明。

现在，我更好地理解我是怎么拼起来可以看到我可能会创建一个补丁证明，将增加如选项 - -exclude-从并行文件，这将允许你指定一个文件，其中包含要从并行测试排除测试文件列表。

更新2012-08-16：我对补丁证明现在，并已提交审查。您可以视图和拉入请求评论。没有总结运行输出后产生。目前还不清楚为什么。

I've got a Perl-based test suite with 10,000+ tests that I would like to make run faster. I've tested using the -j flag to prove, and I have found that most-but-not-all of my tests are ready to run in parallel.

While I can work on making the remaining tests to be "parallel friendly", I expect there always be some tests which are not. What's a good way to manage this? I would like for it to be easy to run the whole set of tests efficiently, and make it easy to mark tests as "not-parallel-ready" if I need to.

Here are some options I see:

1. prove could be patched to support some tests as not-parallel-ready
2. Jenkins is being used to manage the test suite runs. I could split off the non-parallel tests into their own run. In other words, give up and use two test runs.
3. Perhaps there is a way to merge two TAP result streams together that I have yet to recover.

I'm not too concerned with how I will manage the list of exceptions. Either I can keep a list in a file as part of the test harness infrastructure, or I could put something in each test header that would mark it as such, and our test harness could determine the list of exceptions dynamically.

( The test suite is partially based on Test::Class, and I'll also be looking at Test::Class::Load to speed it up as well. )

解决方案

I found a solution. It's in the documentation for aggregate_tests() for TAP::Harness. It includes a code sample for how I could write my own harness for this purpose:

...This is useful, for example, in the case where some tests should run in parallel but others are unsuitable for parallel execution.

my $formatter   = TAP::Formatter::Console->new;
my $ser_harness = TAP::Harness->new( { formatter => $formatter } );
my $par_harness = TAP::Harness->new(
    {   formatter => $formatter,
        jobs      => 9
    }
);
my $aggregator = TAP::Parser::Aggregator->new;

$aggregator->start();
$ser_harness->aggregate_tests( $aggregator, @ser_tests );
$par_harness->aggregate_tests( $aggregator, @par_tests );
$aggregator->stop();
$formatter->summary($aggregator);

From there it looks like I could:

• Sub-class App::Prove and override _runtests(), which is where the new functionality above could be merged in.
• Fork prove so that it calls My::App::Prove instead of App::Prove.

Now that I better understand how the pieces fit together I can see how I might create a patch for prove that would add an option like --exclude-from-parallel FILE, which would allow you to specify a file, which contains a list of test files to be excluded from parallel testing.

UPDATE 2012-08-16: I have a patch for prove now, and have submitted it for review. You can view and comment on the Pull Request. No summary is produced after the run output. It's not clear why.

这篇关于如何运行一个Perl的测试套件并行一些但不是所有的测试？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！