用鼻子测试python多处理池代码 [英] testing python multiprocessing pool code with nose

问题描述

我正在尝试使用鼻子编写的测试，该测试可以进行一些设置 使用多重处理进行计算.

I am trying to write tests with nose that get set up with something calculated using multiprocessing.

我有这个目录结构:

code/
    tests/
        tests.py

tests.py看起来像这样:

tests.py looks like this:

import multiprocessing as mp


def f(i):
    return i ** 2


pool = mp.Pool()
out = pool.map(f, range(10))


def test_pool():
    """Really simple test that relies on the output of pool.map.
    The actual tests are much more complicated, but this is all
    that is needed to produce the problem."""
    ref_out = map(f, range(10))
    assert out == ref_out

if __name__ == '__main__':
    test_pool()

从code目录运行，python tests/tests.py 通过.

nosetests tests/tests.py 失败完成.它启动了，但始终无法通过调用pool.map而挂起.

nosetests tests/tests.py fails to complete. It starts up, but never gets through the call to pool.mapand just hangs.

这是为什么，最简单的解决方案是什么?

Why is this and what is the simplest solution?

推荐答案

该问题与pool.map在全局级别"上被调用有关.通常，您希望避免这种情况，因为即使简单地导入了文件，这些语句也会被执行.

The problem is related to the fact that pool.map is called at the "global level". Normally you want to avoid that, because these statements will be executed even if your file is simply imported.

Nose必须导入模块能够找到您的测试并随后执行它们，因此，我相信导入机制启动时会发生问题(我没有花时间试图找出导致这种行为的确切原因)

Nose has to import your module to be able to find your tests and later execute them, therefore I believe the problem happens while the import mechanism kicks in (I haven't spent time trying to find out the exact reason for this behaviour)

您应该将初始化代码移至测试装置；鼻子使用 with_setup 装饰器来支持灯具.这是一种可能性(将pool和out保留为全局变量时，可能是最简单的更改):

You should move your initialization code to a test fixture instead; Nose supports fixtures with the with_setup decorator. Here is one possibility (probably the simplest change while keeping pool and out as globals):

import multiprocessing as mp
from nose import with_setup

pool = None
out  = None

def f(i):
    return i ** 2

def setup_func():
    global pool
    global out
    pool = mp.Pool()
    out  = pool.map(f, range(10))

@with_setup(setup_func)
def test_pool():
    """Really simple test that relies on the output of pool.map.
    The actual tests are much more complicated, but this is all
    that is needed to produce the problem."""
    global out
    ref_out = map(f, range(10))
    assert out == ref_out

if __name__ == '__main__':
    test_pool()

执行中:

$ nosetests tests/tests.py
.
----------------------------------------------------------------------
Ran 1 test in 0.011s

OK

这篇关于用鼻子测试python多处理池代码的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！