在cython并行中减少数组 [英] Reduction of array in cython parallel

问题描述

我有一个需要包含不同事物之和的数组，因此我想对它的每个元素进行归约. 这是代码:

I have an array that needs to contain sum of different things and therefore I want to perform reduction on each of its elements. Here's the code:

cdef int *a=<int *>malloc(sizeof(int) * 3)
for i in range(3):
    a[i]=1*i
cdef int *b
for i in prange(1000,nogil=True,num_threads=10):
    b=res() #res returns an array initialized to 1s
    with gil: #if commented this line gives erroneous results 
        for k in range(3):
            a[k]+=b[k]
for i in range(3):
    print a[i]

直到带有gil ，代码才能正常运行，否则给出错误的结果. 如何在不使用gil的情况下处理数组每个元素的减少，导致gil我认为gil会阻塞其他线程

Till there is with gil the code runs fine else gives wrong results. How to deal with reductions on each element of array without using gil cause gil i think will block other threads

推荐答案

实际上，减少操作的方法是分别为每个线程求和，然后在最后将它们相加.您可以通过类似

The way reductions usually work in practice is to do the sum individually for each thread, and then add them together at the end. You could do this manually with something like

cdef int *b
cdef int *a_local # version of a that is duplicated by each thread
cdef int i,j,k

# set up as before
cdef int *a=<int *>malloc(sizeof(int) * 3)
for i in range(3):
    a[i]=1*i

# multithreaded from here
with nogil, parallel(num_threads=10):
    # setup and initialise a_local on each thread
    a_local = <int*>malloc(sizeof(int)*3)
    for k in range(3):
        a_local[k] = 0

    for i in prange(1000):
        b=res() # Note - you never free b
                # this is likely a memory leak....

        for j in range(3):
            a_local[j]+=b[j]

    # finally at the end add them all together.
    # this needs to be done `with gil:` to avoid race conditions 
    # but it isn't a problem
    # because it's only a small amount of work being done
    with gil:
        for k in range(3):
            a[k] += a_local[k]
    free(a_local)

这篇关于在cython并行中减少数组的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！