有没有办法控制pytest-xdist如何并行运行测试? [英] Is there a way to control how pytest-xdist runs tests in parallel?

问题描述

我具有以下目录布局:

runner.py
lib/
tests/
      testsuite1/
                 testsuite1.py
      testsuite2/
                 testsuite2.py
      testsuite3/
                 testsuite3.py
      testsuite4/
                 testsuite4.py

testsuite * .py模块的格式如下:

import pytest 
class testsomething:
      def setup_class(self):
          ''' do some setup '''
          # Do some setup stuff here      
      def teardown_class(self):
          '''' do some teardown'''
          # Do some teardown stuff here

      def test1(self):
          # Do some test1 related stuff

      def test2(self):
          # Do some test2 related stuff

      ....
      ....
      ....
      def test40(self):
          # Do some test40 related stuff

if __name__=='__main()__'
   pytest.main(args=[os.path.abspath(__file__)])

我遇到的问题是我想并行执行测试套件"，即我希望testsuite1，testsuite2，testsuite3和testsuite4开始并行执行，但是测试套件中的各个测试需要串行执行.

当我使用py.test中的"xdist"插件并使用"py.test -n 4"启动测试时，py.test会收集所有测试并在4个工作线程之间随机分配测试负载.这导致"setup_class"方法每次在"testsuitex.py"模块中的每次测试中都执行(这违背了我的目的.我希望setup_class每个类仅执行一次，然后在那之后依次执行测试).

基本上我想要执行的样子是:

worker1: executes all tests in testsuite1.py serially
worker2: executes all tests in testsuite2.py serially
worker3: executes all tests in testsuite3.py serially
worker4: executes all tests in testsuite4.py serially

而 worker1, worker2, worker3 and worker4 都是并行执行的.

有没有办法在"pytest-xidst"框架中实现这一目标?

我能想到的唯一选择是启动不同的进程以在Runner.py中分别执行每个测试套件:

def test_execute_func(testsuite_path):
    subprocess.process('py.test %s' % testsuite_path)

if __name__=='__main__':
   #Gather all the testsuite names
   for each testsuite:
       multiprocessing.Process(test_execute_func,(testsuite_path,))

解决方案

您可以使用--dist=loadscope将所有测试归为同一测试类.这是pypi上 pytest-xdist

中的文档

默认情况下，-n选项会将挂起的测试发送到任何可用的工作程序，而没有任何保证的顺序，但是您可以使用以下选项进行控制:

--dist=loadscope:将按模块将测试功能分组，将测试按类分组，然后将测试方法按类进行分组，然后将每个组发送给可用的工作程序，以确保组中的所有测试都在同一过程中运行.如果您有昂贵的模块级或类级灯具，这将很有用.目前无法自定义分组，按类别分组的优先级高于按模块分组的优先级.此功能是在1.19版中添加的.

--dist=loadfile:测试将按文件名分组，然后发送到可用的工作程序，以确保组中的所有测试都在同一工作程序中运行.此功能是在1.21版中添加的.

I have the following directory layout:

runner.py
lib/
tests/
      testsuite1/
                 testsuite1.py
      testsuite2/
                 testsuite2.py
      testsuite3/
                 testsuite3.py
      testsuite4/
                 testsuite4.py

The format of testsuite*.py modules is as follows:

import pytest 
class testsomething:
      def setup_class(self):
          ''' do some setup '''
          # Do some setup stuff here      
      def teardown_class(self):
          '''' do some teardown'''
          # Do some teardown stuff here

      def test1(self):
          # Do some test1 related stuff

      def test2(self):
          # Do some test2 related stuff

      ....
      ....
      ....
      def test40(self):
          # Do some test40 related stuff

if __name__=='__main()__'
   pytest.main(args=[os.path.abspath(__file__)])

The problem I have is that I would like to execute the 'testsuites' in parallel i.e. I want testsuite1, testsuite2, testsuite3 and testsuite4 to start execution in parallel but individual tests within the testsuites need to be executed serially.

When I use the 'xdist' plugin from py.test and kick off the tests using 'py.test -n 4', py.test is gathering all the tests and randomly load balancing the tests among 4 workers. This leads to the 'setup_class' method to be executed every time of each test within a 'testsuitex.py' module (which defeats my purpose. I want setup_class to be executed only once per class and tests executed serially there after).

Essentially what I want the execution to look like is:

worker1: executes all tests in testsuite1.py serially
worker2: executes all tests in testsuite2.py serially
worker3: executes all tests in testsuite3.py serially
worker4: executes all tests in testsuite4.py serially

while worker1, worker2, worker3 and worker4 are all executed in parallel.

Is there a way to achieve this in 'pytest-xidst' framework?

The only option that I can think of is to kick off different processes to execute each test suite individually within runner.py:

def test_execute_func(testsuite_path):
    subprocess.process('py.test %s' % testsuite_path)

if __name__=='__main__':
   #Gather all the testsuite names
   for each testsuite:
       multiprocessing.Process(test_execute_func,(testsuite_path,))

解决方案

You can use --dist=loadscope to group all the tests in the same test class. Here is the doc from pytest-xdist on pypi

By default, the -n option will send pending tests to any worker that is available, without any guaranteed order, but you can control this with these options:

--dist=loadscope: tests will be grouped by module for test functions and by class for test methods, then each group will be sent to an available worker, guaranteeing that all tests in a group run in the same process. This can be useful if you have expensive module-level or class-level fixtures. Currently the groupings can’t be customized, with grouping by class takes priority over grouping by module. This feature was added in version 1.19.

--dist=loadfile: tests will be grouped by file name, and then will be sent to an available worker, guaranteeing that all tests in a group run in the same worker. This feature was added in version 1.21.

这篇关于有没有办法控制pytest-xdist如何并行运行测试?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！