针对多个后端运行 pytest 测试? [英] Running pytest tests against multiple backends?

问题描述

我为与 Github API 交互的代码库构建了一系列测试(使用 pytest)(调用它并接收 webhooks).

I've built a series of tests (using pytest) for a codebase interacting with the Github API (both making calls to it, and receiving webhooks).

目前，这些测试针对 github 的半现实模拟运行:通过 Sentry 的响应拦截对 github 的调用 并运行一个假的 github/git 实现(我需要的位)，它也可以直接从测试用例中进行交互.任何需要触发的 webhook 都使用 Werkzeug 的测试客户端回调到用作 webhook 端点的 WSGI 应用程序.

Currently, these tests run against a semi-realistic mock of github: calls to github are intercepted through Sentry's Responses and run through a fake github/git implementation (of the bits I need), which can also be interacted with directly from the tests cases. Any webhook which needs to be triggered uses Werkzeug's test client to call back into the WSGI application used as webhook endpoint.

这很好用，速度很快(足够)并且是一个很好的默认设置，但是我希望可以选择对 github 本身运行这些相同的测试，这就是我感到困惑的地方:我需要切换当前的实现被测系统(直接库"访问代码库和模拟 github)与不同的系统(API 访问外部"运行代码库和实际 github)，我不太确定如何继续.

This works nicely, fast (enough) and is an excellent default, but I'd like the option to run these same tests against github itself, and that's where I'm stumped: I need to switch out the current implementations of the systems under test (direct "library" access to the codebase & mock github) with different ones (API access to "externally" run codebase & actual github), and I'm not quite sure how to proceed.

我尝试使用 pytest_generate_tests 通过插件切换(代码库和 github 的)夹具实现，但我不太知道这是否可行，到目前为止我的尝试通过 pytest -p 在 pytest 中加载本地文件作为插件并没有取得多大成功.

I attempted to use pytest_generate_tests to switch out the fixture implementations (of the codebase & github) via a plugin but I don't quite know if that would even work, and so far my attempts to load a local file as plugin in pytest via pytest -p <package_name> have not been met with much success.

我想知道我是否朝着正确的方向前进，在这种情况下，是否有人可以帮助在 pytest 中使用本地"插件(不是通过 setuptools 安装的，也不是基于 conftest.py 的).

I'd like to know if I'm heading in the right direction, and in that case if anyone can help with using "local" plugins (not installed via setuptools and not conftest.py-based) in pytest.

不确定这是否有任何相关性，但我使用的是在 CPython 3.6 上运行的 pytest 3.6.0，请求 2.18.4，响应 0.9.0 和 Werkzeug 0.14.1.

Not sure if that has any relevance, but I'm using pytest 3.6.0 running on CPython 3.6, requests 2.18.4, responses 0.9.0 and Werkzeug 0.14.1.

推荐答案

有几种方法可以解决这个问题.默认情况下，我会选择运行您的模拟测试，然后当存在命令行标志时，针对 模拟和真实版本进行测试.

There are several ways to approach this. The one I would go for it by default run your mocked tests and then when a command line flag is present test against both the mock and the real version.

首先是更简单的部分，添加一个命令行选项:

So first the easier part, adding a command line option:

def pytest_addoption(parser):
    parser.addoption('--github', action='store_true',
                 help='Also test against real github')

现在这可以通过 pytestconfig 固定装置作为 pytestconfig.getoption('github') 获得，通常也可以间接获得，例如通过请求夹具作为 request.config.getoption('github').

Now this is available via the pytestconfig fixture as pytestconfig.getoption('github'), often also indirectly available, e.g. via the request fixture as request.config.getoption('github').

现在您需要使用此参数化任何需要与 github API 交互的测试，以便它们与模拟和真实实例一起运行.在不知道您的代码的情况下，Werkzeug 客户端听起来是个好点子:将其变成一个装置，然后可以对其进行参数化以返回您提到的真实客户端或测试客户端:

Now you need to use this parametrize any test which needs to interact with the github API so that they get run both with the mock and with the real instance. Without knowing your code it sounds like a good point would be the Werkzeug client: make this into a fixture and then it can be parameterized to return both a real client or the test client you mention:

@pytest.fixture
def werkzeug_client(werkzeug_client_type):
    if werkzeug_client_type == 'real':
        return create_the_real_client()
    else:
        return create_the_mock_client()

def pytest_generate_tests(metafunc):
    if 'werkzeug_client_type' in metafunc.fixturenames:
        types = ['mock']
        if metafunc.config.getoption('github'):
            types.append('real')
        metafunc.parametrize('werkzeug_client_type', types)

现在，如果您将测试编写为:

Now if you write your test as:

def test_foo(werkzeug_client):
     assert werkzeug_client.whatever()

使用pytest --github 调用时，您将获得一个正常测试和两个测试.

You will get one test normally and two tests when invoked with pytest --github.

(注意钩子必须在 conftest.py 文件中，而夹具可以在任何地方.要特别注意 pytest_addoption 钩子应该只在顶级 conftest 中使用文件以避免混淆 pytest 何时使用钩子，何时不使用.所以你应该把所有这些代码放在一个顶级的 conftest.py 文件中.)

(Be aware hooks must be in conftest.py files while fixtures can be anywhere. Be extra aware that the pytest_addoption hook should really only be used in the toplevel conftest file to avoid you from confusion about when the hook is used by pytest and when not. So you should put all this code in a toplevel conftest.py file really.)

这篇关于针对多个后端运行 pytest 测试?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！