在python中使用鼻子获取测试以并行化 [英] Getting tests to parallelize using nose in python

问题描述

我有一个包含许多.py文件的目录(例如test_1.py，test_2.py等)，每个文件都编写正确，可以与鼻子一起使用.因此，当我运行鼻子测试脚本时，它将在所有.py文件中找到所有测试并执行它们.

I have a directory with lots of .py files (say test_1.py, test_2.py and so on) Each one of them is written properly to be used with nose. So when I run nosetests script, it finds all the tests in all the .py files and executes them.

我现在想并行化它们，以便将所有.py文件中的所有测试视为可并行化并委托给辅助进程.

I now want to parallelize them so that all the tests in all .py files are treated as being parallelizable and delegated to worker processes.

默认情况下，似乎:

nosetests --processes=2 

完全不引入并行性，并且所有.py文件中的所有测试仍仅在一个进程中运行

introduces no parallelism at all and all tests across all .py files still run in just one process

我尝试在每个.py文件中放置_multiprocess_can_split_ = True，但这没什么作用

I tried putting a _multiprocess_can_split_ = True in each of the .py files but that makes no difference

感谢您的投入！

推荐答案

看来，鼻子，实际上是多进程插件，将使测试并行运行.需要注意的是它的工作方式，最终可能无法在多个进程上执行测试.该插件创建一个测试队列，产生多个进程，然后每个进程同时使用该队列.没有为每个进程分配测试，因此，如果您的测试执行得非常快，它们最终可能会在同一进程中执行.

It seems that nose, actually the multiprocess plugin, will make test run in parallel. The caveat is that the way it works, you can end up not executing test on multiple processes. The plugin creates a test queue, spawns multiple processes and then each process consumes the queue concurrently. There is no test dispatch for each process thus if your test are executing very fast, they could end up being executed in the same process.

下面的示例显示此行为:

The following example displays this beaviour:

文件test1.py

File test1.py

import os
import unittest

class testProcess2(unittest.TestCase):

    def test_Dummy2(self):
        self.assertEqual(0, os.getpid())

文件test2.py

File test2.py

import os
import unittest

class testProcess2(unittest.TestCase):

    def test_Dummy2(self):
        self.assertEqual(0, os.getpid())

运行鼻子测试--processes = 2个输出(注意相同的进程ID)

Running nosetests --processes=2 outputs (notice the identical process id)

FF
======================================================================
FAIL: test_Dummy2 (test1.testProcess2)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\temp\test1.py", line 7, in test_Dummy2
    self.assertEqual(0, os.getpid())
AssertionError: 0 != 94048

======================================================================
FAIL: test_Dummy1 (test2.testProcess1)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\temp\test2.py", line 8, in test_Dummy1
    self.assertEqual(0, os.getpid())
AssertionError: 0 != 94048

----------------------------------------------------------------------
Ran 2 tests in 0.579s

FAILED (failures=2)

现在，如果我们在一项测试中添加睡眠

Now if we add a sleep in one of the test

import os
import unittest
import time

class testProcess2(unittest.TestCase):

    def test_Dummy2(self):
        time.sleep(1)
        self.assertEqual(0, os.getpid())

我们得到了(请注意不同的进程ID)

We get (notice the different process id)

FF
======================================================================
FAIL: test_Dummy1 (test2.testProcess1)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\temp\test2.py", line 8, in test_Dummy1
    self.assertEqual(0, os.getpid())
AssertionError: 0 != 80404

======================================================================
FAIL: test_Dummy2 (test1.testProcess2)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\temp\test1.py", line 10, in test_Dummy2
    self.assertEqual(0, os.getpid())
AssertionError: 0 != 92744

----------------------------------------------------------------------
Ran 2 tests in 1.422s

FAILED (failures=2)

这篇关于在python中使用鼻子获取测试以并行化的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！