在不同的进程中运行 py.test 测试 [英] Run py.test test in different process
问题描述
我正在尝试测试 tensorflow 程序.我正在使用参数化的 py.test 夹具设置 tensorflow 会话:
I'm trying to test tensorflow program. I'm setting up tensorflow session using parametrized py.test fixture:
@pytest.fixture(scope="session", params=configuration)
def session(request):
if request.param == 'tensorflow':
return tf.Session()
elif request.param == 'tensorflow-eager':
tfe.enable_eager_execution()
return tf.Session()
elif ...
Tensorflow 具有全局状态,因此多次测试启动可能会污染它.例如,在启用 Eager Execution 后无法禁用它.有没有办法指示 py.test 为每个测试创建一个新进程?或者除了使用参数化夹具之外的另一种为测试配置环境的方法?示例用法:
Tensorflow has global state, thus several test launches can pollute it. For example there is no way to disable eager execution after it has been enabled. Is there a way to instruct py.test to create a new process for each test? Or another way to configure environment for the test besides using parametrized fixture? Example usage:
@pytest.mark.parametrize("bias_type", ['variable', 'ndarray', 'list', 'tuple'])
@pytest.mark.parametrize("kernel_type", ['variable', 'ndarray', 'list', 'tuple'])
@pytest.mark.parametrize("input_type", ['variable', 'ndarray', 'list', 'tuple'])
def test_convolution(session, input_type, kernel_type, bias_type):
...
推荐答案
正如评论中所建议的,使用 pytest-xdist
将是解决方案.该插件专为并行或分布式执行测试而设计(甚至可以进行多平台执行),但非常适合满足您在单独进程中运行每个测试的要求 - 您可以使用 --分叉
参数.
As suggested in the comments, using pytest-xdist
will be the solution. The plugin is designed for parallel or distributed execution of tests (even a multi-platform execution is possible), but is well suited to serve your request of running each test in a separate process - you can achieve this with the --forked
argument.
--forked
参数在 Windows 上不起作用,因为 Windows 不支持 fork-exec 模型并且不提供 fork()
的任何替代品.
--forked
argument will not work on Windows, because Windows doesn't support the fork-exec model and doesn't ship any replacement for fork()
.
让我们定义一个fixture,它会在运行每个测试之前尝试开启急切执行:
Let's define a fixture that will attempt to turn the eager execution on before running each test:
from tensorflow.contrib.eager.python import tfe
import pytest
@pytest.fixture(scope='function', autouse=True)
def eager(request):
tfe.enable_eager_execution()
这个夹具显然会失败所有测试,但第一个因为急切执行只能被打开一次.通过一些虚拟测试:
This fixture will obviously fail all tests but the first one since the eager execution can be turned only once. With some dummy tests:
def test_spam():
assert True
def test_eggs():
assert True
def test_bacon():
assert True
按预期运行普通 pytest
失败:
Running plain pytest
fails as expected:
$ pytest -v
============================== test session starts ================================
platform darwin -- Python 3.6.3, pytest-3.3.1, py-1.5.2, pluggy-0.6.0 -- /Users/hoefling/.virtualenvs/stackoverflow/bin/python3.6
cachedir: .cache
rootdir: /Users/hoefling/projects/private/stackoverflow/so-48234032, inifile:
plugins: forked-0.2, mock-1.6.3, hypothesis-3.44.4
collected 3 items
test_spam.py::test_spam PASSED [ 33%]
test_spam.py::test_eggs ERROR [ 66%]
test_spam.py::test_bacon ERROR [100%]
...
E ValueError: Do not call tfe.enable_eager_execution more than once in the
same process. Note eager-mode methods such as tfe.run() also call
tfe.enable_eager_execution.
...
现在安装pytest-xdist
:
$ pip install pytest-xdist
并重新运行测试:
$ pytest -v --forked
============================== test session starts ================================
platform darwin -- Python 3.6.3, pytest-3.3.1, py-1.5.2, pluggy-0.6.0 -- /Users/hoefling/.virtualenvs/stackoverflow/bin/python3.6
cachedir: .cache
rootdir: /Users/hoefling/projects/private/stackoverflow/so-48234032, inifile:
plugins: forked-0.2, xdist-1.22.0, mock-1.6.3, hypothesis-3.44.4
collected 3 items
test_spam.py::test_spam PASSED [ 33%]
test_spam.py::test_eggs PASSED [ 66%]
test_spam.py::test_bacon PASSED [100%]
============================= 3 passed in 6.09 seconds ============================
测试仍然按顺序运行,但每个测试都在自己的子进程中,因此它们都不会失败.
The tests still run sequentially, but each one in an own subprocess, so none of them fails.
现在您可以开始尝试并行执行,例如
Now you can start experimenting with parallel execution, e.g.
$ pytest -v --forked --numprocesses=auto
等等.请参阅插件文档了解更多信息和更多使用示例.
etc. Refer to plugin docs for more info and more usage examples.
这篇关于在不同的进程中运行 py.test 测试的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!