如何在 py.test 运行中多次重复每个测试? [英] How can I repeat each test multiple times in a py.test run?
问题描述
我想按顺序运行每个选定的 py.test 项目任意次数.
我没有看到任何用于执行此操作的标准 py.test 机制.
I want to run each selected py.test item an arbitrary number of times, sequentially.
I don't see any standard py.test mechanism for doing this.
我尝试在 pytest_collection_modifyitems() 挂钩中执行此操作.我修改了传入的项目列表，以多次指定每个项目.测试项的第一次执行按预期工作，但这似乎给我的代码带来了一些问题.
I attempted to do this in the pytest_collection_modifyitems() hook. I modified the list of items passed in, to specify each item more than once. The first execution of a test item works as expected, but that seems to cause some problems for my code.
此外，我希望每次运行都有一个唯一的测试项对象，因为我在各种报告代码中使用 id (item) 作为键.不幸的是，我找不到任何 py.test 代码来复制测试项，copy.copy() 不起作用，并且 copy.deepcopy() 得到一个例外.
Further, I would prefer to have a unique test item object for each run, as I use id (item) as a key in various reporting code. Unfortunately, I can't find any py.test code to duplicate a test item, copy.copy() doesn't work, and copy.deepcopy() gets an exception.
有人可以建议多次执行测试的策略吗?
Can anybody suggest a strategy for executing a test multiple times?
推荐答案
为了多次运行每个测试，我们将在生成测试时以编程方式参数化每个测试.
In order to run each test a number of times, we will programmatically parameterize each test as the tests are being generated.
首先，让我们添加解析器选项(在其中一个 conftest.py 中包含以下内容):
First, let's add the parser option (include the following in one of your conftest.py's):
def pytest_addoption(parser):
parser.addoption('--repeat', action='store',
help='Number of times to repeat each test')
现在我们添加一个pytest_generate_tests"钩子.这就是魔法发生的地方.
Now we add a "pytest_generate_tests" hook. Here is where the magic happens.
def pytest_generate_tests(metafunc):
if metafunc.config.option.repeat is not None:
count = int(metafunc.config.option.repeat)
# We're going to duplicate these tests by parametrizing them,
# which requires that each test has a fixture to accept the parameter.
# We can add a new fixture like so:
metafunc.fixturenames.append('tmp_ct')
# Now we parametrize. This is what happens when we do e.g.,
# @pytest.mark.parametrize('tmp_ct', range(count))
# def test_foo(): pass
metafunc.parametrize('tmp_ct', range(count))
在没有重复标志的情况下运行:
Running without the repeat flag:
(env) $ py.test test.py -vv
============================= test session starts ==============================
platform darwin -- Python 2.7.5 -- py-1.4.20 -- pytest-2.5.2 -- env/bin/python
collected 2 items
test.py:4: test_1 PASSED
test.py:8: test_2 PASSED
=========================== 2 passed in 0.01 seconds ===========================
使用重复标志运行:
(env) $ py.test test.py -vv --repeat 3
============================= test session starts ==============================
platform darwin -- Python 2.7.5 -- py-1.4.20 -- pytest-2.5.2 -- env/bin/python
collected 6 items
test.py:4: test_1[0] PASSED
test.py:4: test_1[1] PASSED
test.py:4: test_1[2] PASSED
test.py:8: test_2[0] PASSED
test.py:8: test_2[1] PASSED
test.py:8: test_2[2] PASSED
=========================== 6 passed in 0.01 seconds ===========================
进一步阅读:
这篇关于如何在 py.test 运行中多次重复每个测试?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
