我该如何对变异设置数据的代码进行基准测试? [英] How can I benchmark code that mutates the setup data?

问题描述

每次运行iter外部的设置代码时，内置基准测试工具的当前实现似乎都会多次运行iter调用中的代码.当被基准测试的代码修改设置数据时，基准测试代码的后续迭代不再基准相同的事情.

The current implementation of the built-in benchmarking tool appears to run the code inside the iter call multiple times for each time the setup code outside the iter is run. When the code being benchmarked modifies the setup data, subsequent iterations of the benchmarked code are no longer benchmarking the same thing.

作为一个具体示例，我正在对从Vec中删除值的速度进行基准测试:

As a concrete example, I am benchmarking how fast it takes to remove values from a Vec:

#![feature(test)]

extern crate test;

use test::Bencher;

#[bench]
fn clearing_a_vector(b: &mut Bencher) {
    let mut things = vec![1];
    b.iter(|| {
        assert!(!things.is_empty());
        things.clear();
    });
}

这将失败:

test clearing_a_vector ... thread 'main' panicked at 'assertion failed: !things.is_empty()', src/lib.rs:11

对元素执行push的类似基准测试表明，iter闭包执行了将近9.8亿次(取决于闭包的执行速度).如果只有一次运行符合我的预期，而又有数百万人没有运行，那么结果可能会非常误导.

Performing a similar benchmark of pushing an element onto the vector shows that the iter closure was executed nearly 980 million times (depending on how fast the closure is). The results could be very misleading if there's a single run that does what I expect and millions more that don't.

使用Rust每晚1.19.0运行测试(f89d8d184 2017-05-30)

Tests were run with Rust nightly 1.19.0 (f89d8d184 2017-05-30)

推荐答案

查看 pew ，最近发布的用于对锈代码进行基准测试的板条箱.它使您可以进行一次克隆到每个基准测试中的设置，或者通过暂停/恢复基准测试来手动运行设置.

Check out pew, a recently published crate for benchmarking rust code. It allows you to do one time set up that is cloned into every benchmark, or manually run set up by pausing/resuming the benchmark.

该库尚处于初期阶段，但这可能正是您所需要的.永远欢迎捐款.

This library is in very early phases, but it might be what you're looking for. Contributions are always welcome.

这篇关于我该如何对变异设置数据的代码进行基准测试?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！