在每个测试类的JVM中并行收集并运行所有junit测试（按类而不是按方法进行并行化） [英] Collect and run all junit tests in parallel with each test class in its own JVM (parallelization by class, not by method)

问题描述

问题

我有一堆junit测试（很多都使用定制运行程序，例如PowerMockRunner或JUnitParamsRunner），都在一些根包 tests （它们位于不同深度的 tests >的各种子包中）。

喜欢收集包 tests 下的所有测试，并在不同的JVM中并行运行每个测试类。理想情况下，并行化是可配置的，但是number_of_cores的缺省也是完全正确的。请注意，我不想在自己的JVM中运行每个方法，但是每个类都需要运行。

背景

I 'PowerMock结合JUnitParams通过注释 @RunWith（PowerMockRunner.class）和 @PowerMockRunnerDelegate（JUnitParamsRunner.class）为我的许多测试。我有大约9000个单元测试，这些测试在正常的时间内完成，但是我拥有一个8核CPU，并且系统在默认的单次测试一次性运行的情况下严重未充分利用。当我经常运行测试时，额外的时间加起来，我真的想要并行运行测试类。

不幸的是，在很多测试我需要嘲笑静态方法，这是我使用PowerMock的原因之一。

我试过的东西

必须对静态方法进行模拟，这使得它不可能使用像 com.googlecode.junittoolbox.ParallelSuite （这是我的初始解决方案）之类的东西，因为它运行的所有内容都是相同的JVM和静态模拟获取所有交错和混乱。或者至少基于我得到的错误，至少在我看来。

我根本不知道JUnit堆栈，但是在捅过之后，看起来另一个选项可能会尝试编写并注入我自己的 RunnerBuilder - 但我不确定是否可以从中产生另一个JVM进程， RunnerBuilder ，不太可能。我认为适当的解决方案将是某种类型的线束，它可以作为一个gradle任务。

我也发现了一些Android Studio（Intellij's）测试选项，但唯一可用的叉选项是方法这不是我想要的。我目前正在探索这个解决方案，所以也许我会想出来，但我想我会并行地询问社区，因为我还没有太多锁定。

UPDATE ：最后能够让Android Studio（Intellij）使用选项收集我的所有测试Test Kind： All in directory 选项没有执行递归搜索）和选择分叉模式 Class 。但是，这仍然会运行顺序找到的每个测试类，并且我没有看到有关并行化的选项。这与我想要的非常接近，但并不完全......：（

）

解决方案

TENTATIVE 回答

不要让这个答案阻止你发布一个更好的答案。但是，如果我在一个星期内没有看到更好的解决方案，我会将其标记为官方的。

不用使用Intellij（Android Studio）内置的JUnit运行配置，我注意到Android Studio有一堆预构建Gradle任务，其中一些参考测试。但是，那些展示了相同的顺序执行问题。然后我发现了运行并行测试任务使用gradle 并将以下语句添加到我的根目录 build.gradle 文件中：

 子项目{
 tasks.withType（Test）{
 maxParallelForks = Runtime.runtime.availableProcessors（）
} 
} 
  

这很好，我的CPU现在被固定到100％（对于大部分运行，因为未完成的测试类的数量变为<利用处理器显然利用率下降）。

这个解决方案的缺点是它不能与Android Studio（Intellij）漂亮的junit runner UI集成。所以，当gradle任务正在进行时，我无法真正看到测试完成率等。在任务执行结束时，它只是吐出总运行时间和指向HTML生成报告的链接。这是一个小问题，我完全可以接受它，但如果我能弄清楚如何改进使用JUnit runner UI的解决方案，那将是非常好的。

Problem
I've a bunch of junit tests (many with custom runners such as PowerMockRunner or JUnitParamsRunner) all under some root package tests (they are in various subpackages of tests at various depths).

I'd like to collect all the tests under package tests and run each test class in a different JVM, in parallel. Ideally, the parallelization would be configurable, but a default of number_of_cores is totally fine as well. Note that I do not want to run each method in its own JVM, but each class.

Background
I'm using PowerMock combined with JUnitParams via annotations @RunWith(PowerMockRunner.class) and @PowerMockRunnerDelegate(JUnitParamsRunner.class) for many of my tests. I have ~9000 unit tests which complete in an "ok" amount of time but I've an 8-core CPU and the systems is heavily underutilized with the default single-test-at-a-time runner. As I run the tests quite often, the extra time adds up and I really want to run the test classes in parallel.

Note that, unfortunately, in a good number of the tests I need to mock static methods which is part of the reason I'm using PowerMock.

What I've Tried
Having to mock static methods makes it impossible to use something like com.googlecode.junittoolbox.ParallelSuite (which was my initial solution) since it runs everything in the same JVM and the static mocking gets all interleaved and messed up. Or so it seems to me at least based on the errors I get.

I don't know the JUnit stack at all, but after poking around, it appears that another option might be to try to write and inject my own RunnerBuilder -- but I'm not sure if I can even spawn another JVM process from within a RunnerBuilder, unlikely. I think the proper solution would be some kind of harness that lives as a gradle task.

I also JUST discovered some Android Studio (Intellij's) test options but the only available fork option is method which is not what I want. I am currently exploring this solution so perhaps I will figure it out but I thought I'd ask the community in parallel since I haven't had much lock yet.

UPDATE: Finally was able to get Android Studio (Intellij) to collect all my tests using options Test Kind: All in directory (for some reason the package option did not do recursive searching) and picking fork mode Class. However, this still runs each test class found sequentially and there are no options that I see about parallelization. This is so close to what I want but not quite... :(

解决方案

TENTATIVE Answer

Don't let this answer discourage you from posting a better answer. However, if I do not see a better solution within a week or so I will mark this as the official one.

Instead of using Intellij's (Android Studio) built-in JUnit run configurations, I noticed that Android Studio has a bunch of pre-build gradle tasks some of which refer to testing. Those however, exhibited the same sequential execution problem. I then found Run parallel test task using gradle and added the following statement to my root build.gradle file:

subprojects {
    tasks.withType(Test) {
        maxParallelForks = Runtime.runtime.availableProcessors()
    }
}

This works great, my CPU is now pegged to 100% (for most of the run, as the number of outstanding test classes becomes < avail processors obviously utilization goes down).

The downside to this solution is that it does not integrate with Android Studio's (Intellij) pretty junit runner UI. So while the gradle task is progressing I cannot really see the rate of test completion, etc. At the end of the task execution, it just spits out the total runtime and a link to an HTML generated report. This is a minor point and I can totally live with it, but it would be nice if I could figure out how to improve the solution to use the the JUnit runner UI.

这篇关于在每个测试类的JVM中并行收集并运行所有junit测试（按类而不是按方法进行并行化）的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！