Python多处理Numpy随机 [英] Python Multiprocessing Numpy Random

问题描述

在多处理调用的函数中，numpy ndarray函数的范围是否有所不同?这是一个示例:

Does the scope of a numpy ndarray function differently within a function called by multiprocessing? Here is an example:

使用python的多处理模块，我正在这样调用一个函数:

Using python's multiprocessing module I am calling a function like so:

for core in range(cores):
    #target could be f() or g()
    proc = mp.Process(target=f, args=(core))
    jobs.append(proc)
for job in jobs:
    job.start()
for job in jobs:
    job.join()

def f(core):
    x = 0
    x += random.randint(0,10)
    print x

def g(core):
    #Assume an array with 4 columns and n rows
    local = np.copy(globalshared_array[:,core])
    shuffled = np.random.permutation(local)

调用f(core)时，x变量在进程本地，即.它会按预期打印一个不同的随机整数.这些值永远不会超过10，表示每个进程中的x=0.正确吗?

Calling f(core), the x variable is local to the process, ie. it prints a different, random integer as expected. These never exceed 10, indicating that x=0 in each process. Is that correct?

调用g(core)并排列数组的副本将返回4个相同的改组"数组.这似乎表明工作副本不是本地子进程.那是对的吗?如果是这样，除了需要使用共享内存空间之外，当需要从共享内存空间填充子进程时，是否可以使ndarray在子进程本地?

Calling g(core) and permuting a copy of the array returns 4 identically 'shuffled' arrays. This seems to indicate that the working copy is not local the child process. Is that correct? If so, other than using sharedmemory space, is it possible to have an ndarray be local to the child process when it needs to be filled from shared memory space?

更改g(core)以添加随机整数似乎具有所需的效果.数组显示不同的值. permutation中一定发生某种事情，对列(每个子进程本地)进行随机排序具有相同的...想法吗?

Altering g(core) to add a random integer appears to have the desired effect. The array's show a different value. Something must be occurring in permutation that is randomly ordering the columns (local to each child process) the same...ideas?

def g(core):
    #Assume an array with 4 columns and n rows
    local = np.copy(globalshared_array[:,core])
    local += random.randint(0,10)

编辑II: np.random.shuffle也表现出相同的行为.数组的内容在改组，但在每个核心上改组为相同的值.

EDIT II: np.random.shuffle also exhibits the same behavior. The contents of the array are shuffling, but are shuffling to the same value on each core.

推荐答案

调用g(core)并置换数组的副本将返回4个相同的改组"数组.这似乎表明工作副本不是本地子进程.

Calling g(core) and permuting a copy of the array returns 4 identically 'shuffled' arrays. This seems to indicate that the working copy is not local the child process.

这可能表明在每个子进程中随机数生成器均被初始化，从而产生相同的序列.您需要为每个孩子的生成器添加种子(也许将孩子的进程ID放入混合中).

What it likely indicates is that the random number generator is initialized identically in each child process, producing the same sequence. You need to seed each child's generator (perhaps throwing the child's process id into the mix).

这篇关于Python多处理Numpy随机的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！