如果一个测试失败了，如何跳过课程中的其余测试? [英] How to skip the rest of tests in the class if one has failed?

问题描述

我正在使用 Jenkins、Python、Selenium2(webdriver) 和 Py.test 框架为网络测试创建测试用例.

I'm creating the test cases for web-tests using Jenkins, Python, Selenium2(webdriver) and Py.test frameworks.

到目前为止，我按照以下结构组织我的测试:

So far I'm organizing my tests in the following structure:

每个类都是测试用例，每个test_方法都是一个测试步骤强>.

each Class is the Test Case and each test_ method is a Test Step.

当一切正常时，此设置非常有效，但是当一个步骤崩溃时，其余的测试步骤"会变得疯狂.在 teardown_class() 的帮助下，我能够将失败包含在类(测试用例)中，但是我正在研究如何改进这一点.

This setup works GREAT when everything is working fine, however when one step crashes the rest of the "Test Steps" go crazy. I'm able to contain the failure inside the Class (Test Case) with the help of teardown_class(), however I'm looking into how to improve this.

如果其中一个失败，我需要以某种方式跳过(或 xfail)一个类中的其余 test_ 方法，以便其余的测试用例不会运行并标记为失败(因为那会是误报)

What I need is somehow skip(or xfail) the rest of the test_ methods within one class if one of them has failed, so that the rest of the test cases are not run and marked as FAILED (since that would be false positive)

谢谢！

更新:我不是在寻找或回答这是不好的做法"，因为这样称呼它很有争议.(每个测试类都是独立的 - 这应该足够了).

UPDATE: I'm not looking or the answer "it's bad practice" since calling it that way is very arguable. (each Test Class is independent - and that should be enough).

更新 2: 在每个测试方法中放置if"条件不是一个选项 - 是很多重复的工作.我正在寻找的是(也许)有人知道如何使用 hooks 到类方法.

UPDATE 2: Putting "if" condition in each test method is not an option - is a LOT of repeated work. What I'm looking for is (maybe) somebody knows how to use the hooks to the class methods.

推荐答案

我喜欢一般的测试步骤"想法.我将其称为增量"测试，恕我直言，它在功能测试场景中最有意义.

I like the general "test-step" idea. I'd term it as "incremental" testing and it makes most sense in functional testing scenarios IMHO.

这是一个不依赖于 pytest 内部细节的实现(官方钩子扩展除外).将其复制到您的 conftest.py 中:

Here is a an implementation that doesn't depend on internal details of pytest (except for the official hook extensions). Copy this into your conftest.py:

import pytest

def pytest_runtest_makereport(item, call):
    if "incremental" in item.keywords:
        if call.excinfo is not None:
            parent = item.parent
            parent._previousfailed = item

def pytest_runtest_setup(item):
    previousfailed = getattr(item.parent, "_previousfailed", None)
    if previousfailed is not None:
        pytest.xfail("previous test failed (%s)" % previousfailed.name)

如果您现在有这样的test_step.py":

If you now have a "test_step.py" like this:

import pytest

@pytest.mark.incremental
class TestUserHandling:
    def test_login(self):
        pass
    def test_modification(self):
        assert 0
    def test_deletion(self):
        pass

然后运行它看起来像这样(使用 -rx 报告 xfail 原因):

then running it looks like this (using -rx to report on xfail reasons):

(1)hpk@t2:~/p/pytest/doc/en/example/teststep$ py.test -rx
============================= test session starts ==============================
platform linux2 -- Python 2.7.3 -- pytest-2.3.0.dev17
plugins: xdist, bugzilla, cache, oejskit, cli, pep8, cov, timeout
collected 3 items

test_step.py .Fx

=================================== FAILURES ===================================
______________________ TestUserHandling.test_modification ______________________

self = <test_step.TestUserHandling instance at 0x1e0d9e0>

    def test_modification(self):
>       assert 0
E       assert 0

test_step.py:8: AssertionError
=========================== short test summary info ============================
XFAIL test_step.py::TestUserHandling::()::test_deletion
  reason: previous test failed (test_modification)
================ 1 failed, 1 passed, 1 xfailed in 0.02 seconds =================

我在这里使用xfail"是因为跳过是针对错误的环境或缺少依赖项、错误的解释器版本.

I am using "xfail" here because skips are rather for wrong environments or missing dependencies, wrong interpreter versions.

请注意，您的示例和我的示例都不能直接用于分布式测试.为此，pytest-xdist 插件需要开发一种方法来定义组/类，以批量发送给一个测试从站，而不是当前模式，通常将一个类的测试功能发送到不同的从站.

Note that neither your example nor my example would directly work with distributed testing. For this, the pytest-xdist plugin needs to grow a way to define groups/classes to be sent whole-sale to one testing slave instead of the current mode which usually sends test functions of a class to different slaves.

这篇关于如果一个测试失败了，如何跳过课程中的其余测试?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！