自定义pytest junitxml失败报告 [英] Customizing pytest junitxml failure reports

问题描述

我正在尝试内省测试失败，并将其他数据包含到junit xml测试报告中.具体来说，这是对外部产品的一组功能测试，我想将产品的日志包含在故障报告中.

I am trying to introspect test failures and include additional data into the junit xml test report. Specifically, this is a suite of functional tests on an external product, and I want to include the product's logs into the failure reports.

使用找到的方法这里，我能够在执行多次调用之前将日志打印到stdout，最终显示在詹金的失败报告中.但是我敢肯定，有一种更好的方法可以实现这一目标.

Using the method found here, I was able to print the logs to stdout before executing the multicall which eventually show in jenkin's fail report. But I'm sure there's a better way to achieve this.

我试图使用pytest_runtest_logreport钩子将日志附加到"sections"属性中，该属性已经包含捕获的stdout"和捕获的stderr"流.但是新添加的部分不会将其添加到xml文件中.我也直接在pytest_runtest_makereport钩子中尝试了上述技术，结果相似.

I tried to use the pytest_runtest_logreport hook to append the logs into the 'sections' attribute, which already contains the 'captured stdout' and 'captured stderr' streams. But the newly added sections do not make it to the xml file. I tried the above technique directly into the pytest_runtest_makereport hook as well, with similar results.

pytest 2.7的发行说明指出，不再支持2.8的多调用支持，而@ pytest.mark.hookwrapper是实现此功能的新方法，但是我似乎根本无法使它起作用-yield"返回None而不是CallOutcome对象(在makereport挂钩中对其进行了尝试).而且即使返回了某些内容，我也不确定是否可以向其中添加可以在xml报告中显示的内容.

The release notes for pytest 2.7 states that using multicall support is being dropped for 2.8 and that @pytest.mark.hookwrapper is the new way to do it, however I can't seem to make that work at all - the "yield" returns None instead of a CallOutcome object (tried it in the makereport hook). And even if it returned something, I'm not sure I could add stuff to it that would show up in the xml report.

我缺少任何可以灵活执行此功能的功能吗?(我的意思是灵活:不绑定到stdout或记录捕获日志插件之类的调用)

Is there any functionality I'm missing that will let me do this in a flexible way? (by flexible I mean: not being bound to stdout or logging calls like the capture-logs plugin)

推荐答案

由于我需要访问测试项目的真菌(和测试结果)以进行报告，因此我能够将逻辑移至 pytest_runtest_makereport(项目，__ multicall __)钩子.诀窍是执行多次调用，该调用将返回报告对象:

Since I needed access to the test item's funcargs (and test result) for my reporting, I was able to move the logic to the pytest_runtest_makereport(item, __multicall__) hook. The trick is to execute the multicall, which returns the report object:

@pytest.mark.tryfirst
def pytest_runtest_makereport(item, call, __multicall__):
    report = __multicall__.execute()
    # then I was able to manipulate report and get the same results as below

---

Bruno的回答给了我更彻底地分析此功能所需的动力:)

---

Bruno's answer gave me the motivation I needed to analyze this feature more thoroughly :)

这是它的工作方式:

def pytest_runtest_logreport(report):
    if report.failed:
        report.longrepr.sections.append(("Header", "Message", "-"))
        report.sections.append(("Captured stdout", "This is added to stdout"))
        report.sections.append(("Captured stderr", "This is added to stderr"))
        report.sections.append(("Custom Section", "This can only be seen in the console - the xml won't have it."))

longrepr 属性仅在失败的情况下可用.它需要一个三元组，最后一个值是用于装饰标头/包围标头的字符.它将显示在报告的失败"部分:

The longrepr attribute is only available in case of failures. It takes a 3-tuple, the last value being a character used to decorate the decorate/surround the header. It will appear in the "failure" section of the report:

----------------------------------- Header ------------------------------------
Message

自定义部分将创建其他结果部分，以打印到 控制台 中.但他们不会将其添加到junitxml中:

Custom sections will create additional result sections to be printed out to the console. But they won't make it to junitxml:

------------------------------- Custom Section --------------------------------
This can only be seen in the console - the xml won't have it.

junitxml报告只有两个部分:out和err.要向其中添加自定义文本，您必须创建名为"Captured std"的部分，只有这些部分才能将其添加到xml文件中.任何其他名称都将导致一个自定义部分，该部分只能在控制台中看到.

The junitxml report only has 2 sections: out and err. To add custom text to it, you must create sections called "Captured std" and only those will make it to the xml file. Any other name will result in a custom section that will only be seen in the console.

这是使用上面的代码生成的junitxml，为了便于这篇文章，对它们进行了一些重新格式化:

Here's the resulting junitxml using the code above, with some reformatting for the sake of this post:

<?xml version="1.0" encoding="utf-8" ?> 
<testsuite errors="0" failures="1" name="pytest" skips="0" tests="1" time="0.646">
  <testcase classname="test_reporting" name="test_fail" time="0.000999927520752">
    <failure message="test failure">
      @ut def test_fail(): > assert 0, "It failed"
      E AssertionError: It failed 
      E assert 0 test_reporting.py:346: AssertionError
      ----------------------------------- Header ------------------------------------
      Message
    </failure> 
    <system-out>This is added to stdout</system-out> 
    <system-err>This is added to stderr</system-err> 
  </testcase>
</testsuite>

这篇关于自定义pytest junitxml失败报告的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！