运行自动发现的 python 单元测试的子集 [英] Running subset of auto-discovered python unittests

问题描述

简短问题
是否可以在运行时选择在 Python 的 unittest 模块中使用自动发现方法时将要运行的单元测试.

Short Question
Is it possible to select, at run time, what unittests are going to be run when using an auto-discovery method in Python's unittest module.

背景
我正在使用 unittest 模块在外部系统上运行系统测试.请参阅下面的示例 sudo-testcase.unittest 模块允许我创建任意数量的测试用例，我可以使用 unittest 的 testrunner 运行这些测试用例.我一直在使用这种方法大约 6 个月的持续使用，并且效果非常好.

此时我想尝试使其更加通用和用户友好.对于我现在正在运行的所有测试套件，我已经硬编码了必须为每个系统运行哪些测试.这对于未经测试的系统来说很好，但是当测试错误地失败(用户连接到错误的测试点等)时，他们必须重新运行整个测试套件.由于一些完整的套房可能需要长达 20 分钟的时间，这远非理想之选.

At this point in time I am wanting to try and make this more generic and user friendly. For all of my test suites that I am running now, I have hard coded which tests must run for every system. This is fine for an untested system, but when a test fails incorrectly (a user connects to the wrong test point etc...) they must re-run the entire test suite. As some of the complete suites can take up to 20 min, this is no where near ideal.

我知道可以创建自定义测试套件构建器来定义要运行的测试.我的问题是有数百个测试用例可以运行，如果测试用例名称发生变化等，维护这将是一场噩梦......

I know it is possible to create custom testsuite builders that could define which tests to run. My issue with this is that there are hundreds of testcases that can be run and maintaining this would be come a nightmare if test case names change etc...

我希望使用 nose 或内置的 unittest 模块来实现这一点.对于这两个选项，发现部分似乎非常简单，但我的问题是，选择要运行的测试用例子集的唯一方法是定义测试用例名称中存在的模式.这意味着我仍然需要硬编码模式列表来定义要运行的测试用例.因此，如果我必须对这个列表进行硬编码，那么使用自动发现有什么意义(请注意这是一个反问句)?

My hope was to use nose, or the built-in unittest module to achieve this. The discovery part seems to be pretty straight forward for both options, but my issue is that only way to select a subset of testcases to run is to define a pattern that exists in the testcase name. This means I would still have to hard code a list of patterns to define what testcases to run. So if I have to hard code this list, what is the point of using the auto-discovery (please note this is rhetorical question)?

我的最终目标是有一种通用的方式来选择在执行期间运行哪些单元测试，以用户可以编辑的复选框或文本字段的形式.理想情况下，该解决方案将使用 Python 2.7，并且需要在 Windows、OSX 和 Linux 上运行.

My end goal is to have a generic way to select which unittests to run during execution, in the form of check boxes or a text field the user can edit. Ideally the solution would be using Python 2.7 and need would need to run on Windows, OSX, and Linux.

编辑
为了帮助澄清，我不希望该工具生成选项列表或复选框.理想情况下，该工具将返回目录中所有测试的列表，包括完整路径和测试用例所属的套件(如果有).使用此列表，我将构建用户与之交互的复选框或组合框，并将这些测试即时传递到测试套件中以运行.

Edit
To help clarify, I do not want the tool to generate the list of choices or the check boxes. The tool ideally would return a list of all of the tests in a directory, including the full path and what Suite (if any) the testcase belongs down. With this list, I would build the check boxes or a combo box the user interacts with and pass these tests into a testsuite on the fly to run.

示例测试

test_measure_5v_reference 
    1) Connect to DC power supply via GPIB
    2) Set DC voltage to 12.0 V
    3) Connect to a Digital Multimeter via GPIB
    4) Measure / Record the voltage at a given 5V reference test point
    5) Use unittest's assert functions to make sure the value is within tolerance

推荐答案

将每个测试子集存储在其自己的模块中.通过让用户使用(如您所述)复选框或文本输入来选择模块名称来获取模块名称列表.获得模块名称列表后，您可以构建相应的测试套件，执行类似于以下操作.

Store each subset of tests in its own module. Get a list of module names by having the user select them using, as you stated, checkboxes or text entry. Once you have the list of module names, you can build a corresponding test suite doing something similar to the following.

testsuite = unittest.TestSuite()
for module in modules:
  testsuite.addTest(unittest.defaultTestLoader.loadTestsFromModule(module))

这篇关于运行自动发现的 python 单元测试的子集的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！