pytest-hypothesis 运行之间的测试隔离 [英] test isolation between pytest-hypothesis runs

问题描述

我刚刚将 pytest 测试套件从 quickcheck 迁移到 hypothesis.这很有效(并立即发现了一些隐藏的边缘情况错误)，但我看到的一个主要区别与两个属性管理器之间的测试隔离有关.

I just migrated a pytest test suite from quickcheck to hypothesis. This worked quite well (and immediately uncovered some hidden edge case bugs), but one major difference I see is related to test isolation between the two property managers.

quickcheck 似乎只是用不同的参数值多次运行测试函数，每次运行我的函数范围的装置.这也会导致 pytest 的输出中出现更多的点.

quickcheck seems to simply run the test function multiple times with different parameter values, each time running my function-scoped fixtures. This also results in many more dots in pytest's output.

hypothesis 然而，似乎只多次运行测试函数的主体，这意味着例如在每次运行之间没有事务回滚.这意味着当我的测试将某些内容插入数据库时​​，我无法可靠地断言许多数据库条目，因为上次运行的所有条目仍然会挂起.

hypothesis however seems to run only the body of the test function multiple times, meaning for example no transaction rollbacks between individual runs. This then means I cannot reliably assert for a number of DB entries when my test inserts something into the DB for example, since all the entries from the previous run would still be hanging around.

我在这里遗漏了一些明显的东西还是这是预期的行为?如果是这样，有没有办法将假设完成的运行次数作为在测试中使用的变量?

Am I missing something obvious here or is this expected behaviour? If so, is there a way to get the number of runs hypothesis has done as a variable to use inside the test?

推荐答案

恐怕你有点卡住了，目前没有任何好的解决方案.

I'm afraid you're a bit stuck and there isn't currently any good solution to this problem.

Hypothesis 需要工作的方式(这是它对 pytest-quickcheck 的许多改进的来源)不符合 pytest 关于测试执行的假设.问题是 主要在 pytest 方面 - 当前的 pytest 夹具系统有一些关于如何运行测试的假设非常成熟，但不能很好地控制测试执行，上次我试图解决这个问题时，我最终在放弃之前投入了大约一周的工作，基本上说要么需要在 pytest 方面做出改变，要么有人需要资助这项工作如果它会变得更好.

The way Hypothesis needs to work (which is the source of a lot of its improvements over pytest-quickcheck) doesn't meet pytest's assumptions about test execution. The problem is mostly on the pytest side - the current pytest fixture system has some very baked in assumptions about how you run a test that do not play well with taking control of the test execution, and the last time I tried to work around this I ended up sinking about a week of work into it before giving up and basically saying that either something needs to change on the pytest side or someone needs to fund this work if it's going to get any better.

这篇关于pytest-hypothesis 运行之间的测试隔离的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！