使用 pytest 对参数化基准进行分组 [英] Grouping Parametrized Benchmarks with pytest
问题描述
我目前正在对我使用
有谁知道我可以对输出进行分组的方法,但可以说是我示例中的 case
?或者更好的是,通过 (case,n)
元组?
一个有用的评论 说 pytest 的 master 分支正在支持这个确切的功能,但我无法让它工作(下一个版本交叉手指).>
与此同时,我想出了这个方便的解决方法.我可以使用这种方法按 case
分组,但不能按 (case,n)
分组.我在每个测试用例上方添加了一个 @benchmark_this
装饰器来包装 benchmark
调用.即使没有按测试用例分组的额外好处,它也非常方便!
def benchmark_this(test):定义包装器(基准,t,n):基准(测试,无,t,n)返回包装器类型 = [BaseTree, AvlTree]尺寸 = [100,300,1000]@pytest.mark.parametrize('t', 类型)@pytest.mark.parametrize('n', 尺寸)@benchmark_thisdef test_insertRandomOrder(benchmark, t, n):随机种子(0x1C2C6D66)树 = t()对于范围(n)中的我:tree.insert(random.randint(0, 0x7FFFFFFF), i)@pytest.mark.parametrize('t', 类型)@pytest.mark.parametrize('n', 尺寸)@benchmark_thisdef test_insertDescendingOrder(benchmark, t, n):树 = t()对于范围(n)中的我:tree.insert(n-i, i)# ...
用
调用<块引用>py.test --benchmark-group-by=func
I'm currently benchmarking an implementation of an AVL Tree I made against a non-rebalancing binary search tree using pytest-benchmark. It seems to be working well for me so far but I've run into an issue. In order to consolidate the test file, I found that I can parametrize tests and also group the output of the benchmark for readability but I can't seem to do them both at the same time.
My current insertion benchmark:
# always the same for repeatability
random.seed(0x1C2C6D66)
def insertRandomOrder(t, n):
tree = t()
for i in range(0,n):
tree.insert(random.randint(0,0x7FFFFFFF),i)
def insertDescendingOrder(t, n):
tree = t()
for i in range(0,n):
tree.insert(n-i,i)
def insertOutInOrder(t, n):
tree = t()
for i in range(0,n):
idx = (i%2)*n + (1-2*(i%2))*i
tree.insert(idx,i)
def insertAscendingOrder(t, n):
tree = t()
for i in range(0,n):
tree.insert(i,i)
types = [BaseTree, AvlTree]
sizes = [100,300,1000]
cases = [insertAscendingOrder, insertDescendingOrder, insertOutInOrder, insertRandomOrder]
@pytest.mark.parametrize('t', types)
@pytest.mark.parametrize('n', sizes)
@pytest.mark.parametrize('case', cases)
def test_insert_benchmark(benchmark, t, n, case):
benchmark(case, t, n)
And here is the output:
Does anyone know of a way where I could group the output but say, the case
from my example? Or better yet, by a (case,n)
tuple?
There was a useful comment saying that the master branch of pytest is in the process of supporting this exact feature, but I was unable to get it to work (fingers crossed for next release).
In the meantime, I figured out this handy work around. I'm able to group by case
, but not by (case,n)
with this method. I added a @benchmark_this
decorator above each test case to wrap the benchmark
call. It's pretty handy even without the extra benefit of grouping by test case!
def benchmark_this(test):
def wrapper(benchmark, t, n):
benchmark(test, None, t, n)
return wrapper
types = [BaseTree, AvlTree]
sizes = [100,300,1000]
@pytest.mark.parametrize('t', types)
@pytest.mark.parametrize('n', sizes)
@benchmark_this
def test_insertRandomOrder(benchmark, t, n):
random.seed(0x1C2C6D66)
tree = t()
for i in range(n):
tree.insert(random.randint(0, 0x7FFFFFFF), i)
@pytest.mark.parametrize('t', types)
@pytest.mark.parametrize('n', sizes)
@benchmark_this
def test_insertDescendingOrder(benchmark, t, n):
tree = t()
for i in range(n):
tree.insert(n-i, i)
# ...
Invoked with
py.test --benchmark-group-by=func
这篇关于使用 pytest 对参数化基准进行分组的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!