JMH：在所有Benchmark测试中使用相同的静态对象 [英] JMH: Using the same static object in all Benchmark tests

问题描述

我有一个构造一些复杂数据的类（想象一下大型XML或JSON结构 - 那种事情）。构建它需要时间。所以我想构建一次，然后在所有测试中使用相同的数据。目前我基本上在定义 main 的类中定义了 public static 对象实例，然后在测试（代码是一个非常简单的例子）：

I have a class that constructs some complicated data (imagine a large XML or JSON structure - that sort of thing). Constructing it takes time. So I want to construct it once, and then use that same data in all tests. Currently I basically have a public static object instance defined in a class that defines main, and then refer to it explicitly in the tests (code is a very simplified example):

public class Data 
{
    // This class constructs some complicated data 
}

public class TestSet 
{
    public static final Data PARSE_ME = new Data(...);

    public static void main(String[] args) throws RunnerException 
    {
        Options opt = new OptionsBuilder()
                .include(".*ParserTest") // several tests
                .forks(1)
                .build();

        new Runner(opt).run();
    }
}

@State(Scope.Thread)
public class SomeParserTest
{
    @Setup(Level.Iteration)
    public void setup()
    {
        Parser parser = new Parser(TestSet.PARSE_ME);
    }

    @Benchmark
    public void getId() 
    {
        parser.getId(123);
    }
}

这当然很可怕...同样evil选项只是创建一个单独的类，以便它可以容纳一个静态对象。使用类似

And this is awful of course... An equally evil option would be creating a separate class just so that it can hold a single static object. It would be nice to use something like

Options opt = new OptionsBuilder()
    ...
    .param(/*my Data object comes here*/)

但是 param 只接受字符串，所以不确定如何传递一个对象（更重要的是：对象的相同实例！）。

but param only accepts Strings, so not sure how would I pass an object (and more importantly: the same instance of the object!) to it.

那么，除了上面描述的全球物体之外，还有什么更优雅的东西吗？

So is there anything more elegant, than a global object, I described above?

推荐答案

不幸的是，JMH无法在基准测试之间共享数据。

Unfortunately, JMH provides no way to share data between the benchmarks.

首先，当一个基准测试可以默默地修改另一个基准测试的输入数据时，这会破坏基准测试隔离，从而导致比较不正确。这就是为什么每个基准测试都需要 @Setup @State 对象。

For one thing, this breaks benchmark isolation, when one benchmark can silently modify the input data for another benchmark, rendering the comparison incorrect. This is why you are required to @Setup the @State object for every benchmark.

但更重要的是，无论你构建什么技巧来共享基准测试之间的数据（例如 static 都可以从两者访问）都会破坏默认值分叉模式，当JMH将在自己的VM中执行每个测试时。值得注意的是，你建议用静态最终Data TestSet.PARSE_ME 建议实际执行每个 @Benchmark ，因为每个无论如何，新的VM实例必须初始化 TestSet ;）当然，你可以禁用分叉，但这会带来比它解决的更多问题。

But more importantly, whatever trick you build to share the data between the benchmarks (e.g. static field accessible from both) would break in the default "forked" mode, when JMH will execute each test in its own VM. Notably, the thing you are suggesting with static final Data TestSet.PARSE_ME would actually execute for each @Benchmark, since every new VM instance would have to initialize TestSet anyhow ;) Granted, you may disable forking, but that introduces more problems than it solves.

因此，花时间让设置成本更容易忍受可能是一个更好的主意，因此它不会令人痛苦。例如，从磁盘反序列化数据而不是计算它。或者，想出一种更快的计算方法。

Therefore, it may be a better idea to invest time into making the setup costs more tolerable, so that it is not excruciatingly painful. E.g., deserialize the data from disk instead of computing it. Or, just come up with a faster way to compute.

这篇关于JMH：在所有Benchmark测试中使用相同的静态对象的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！