如何在 Perl 测试套件中并行运行一些但不是所有测试? [英] How to run some but not all tests in a Perl test suite in parallel?

问题描述

我有一个基于 Perl 的测试套件，其中包含 10,000 多个测试，我想让它们运行得更快.我已经使用 -j 标志对 prove 进行了测试，并且我发现我的大多数但不是所有测试都准备好并行运行.

I've got a Perl-based test suite with 10,000+ tests that I would like to make run faster. I've tested using the -j flag to prove, and I have found that most-but-not-all of my tests are ready to run in parallel.

虽然我可以努力使其余测试并行友好"，但我希望总有一些测试不是.有什么好的方法来管理这个?我希望它能够轻松高效地运行整个测试集，并在需要时轻松将测试标记为未准备好".

While I can work on making the remaining tests to be "parallel friendly", I expect there always be some tests which are not. What's a good way to manage this? I would like for it to be easy to run the whole set of tests efficiently, and make it easy to mark tests as "not-parallel-ready" if I need to.

以下是我看到的一些选项:

Here are some options I see:

1. 证明可以进行修补以支持一些未准备好并行的测试
2. Jenkins 被用于管理测试套件的运行.我可以将非并行测试拆分为他们自己的运行.换句话说，放弃并使用两次测试运行.
3. 也许有一种方法可以将两个我尚未恢复的 TAP 结果流合并在一起.

1. prove could be patched to support some tests as not-parallel-ready
2. Jenkins is being used to manage the test suite runs. I could split off the non-parallel tests into their own run. In other words, give up and use two test runs.
3. Perhaps there is a way to merge two TAP result streams together that I have yet to recover.

我不太关心如何管理例外列表.我可以在文件中保留一个列表作为测试工具基础设施的一部分，或者我可以在每个测试标头中放置一些东西来标记它，我们的测试工具可以动态确定异常列表.

I'm not too concerned with how I will manage the list of exceptions. Either I can keep a list in a file as part of the test harness infrastructure, or I could put something in each test header that would mark it as such, and our test harness could determine the list of exceptions dynamically.

(测试套件部分基于 Test::Class，我还将研究 Test::Class::Load 以加快它的速度.)

( The test suite is partially based on Test::Class, and I'll also be looking at Test::Class::Load to speed it up as well. )

推荐答案

我找到了解决方案.它在 aggregate_tests() for TAP::Harness 的文档中.它包含一个代码示例，说明我如何为此目的编写自己的工具:

I found a solution. It's in the documentation for aggregate_tests() for TAP::Harness. It includes a code sample for how I could write my own harness for this purpose:

...这很有用，例如，在某些测试应该并行运行，但其他不适合并行执行.

...This is useful, for example, in the case where some tests should run in parallel but others are unsuitable for parallel execution.

my $formatter   = TAP::Formatter::Console->new;
my $ser_harness = TAP::Harness->new( { formatter => $formatter } );
my $par_harness = TAP::Harness->new(
    {   formatter => $formatter,
        jobs      => 9
    }
);
my $aggregator = TAP::Parser::Aggregator->new;

$aggregator->start();
$ser_harness->aggregate_tests( $aggregator, @ser_tests );
$par_harness->aggregate_tests( $aggregator, @par_tests );
$aggregator->stop();
$formatter->summary($aggregator);

从那里看起来我可以:

• 子类 App::Prove 并覆盖 _runtests()，这是可以合并上述新功能的地方.
• Fork prove 使其调用 My::App::Prove 而不是 App::Prove.

• Sub-class App::Prove and override _runtests(), which is where the new functionality above could be merged in.
• Fork prove so that it calls My::App::Prove instead of App::Prove.

现在我更好地理解了这些部分是如何组合在一起的，我可以看到如何为 prove 创建一个补丁，该补丁将添加一个选项，如 --exclude-from-parallel FILE，这将允许您指定一个文件，其中包含要从并行测试中排除的测试文件列表.

Now that I better understand how the pieces fit together I can see how I might create a patch for prove that would add an option like --exclude-from-parallel FILE, which would allow you to specify a file, which contains a list of test files to be excluded from parallel testing.

2012-08-16 更新:我现在有一个 prove 补丁，并已提交以供审核.您可以查看和评论拉取请求.运行输出后不生成摘要.目前尚不清楚为什么.

UPDATE 2012-08-16: I have a patch for prove now, and have submitted it for review. You can view and comment on the Pull Request. No summary is produced after the run output. It's not clear why.

这篇关于如何在 Perl 测试套件中并行运行一些但不是所有测试?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！