Cython并行Prange-线程局部性? [英] Cython parallel prange - thread locality?
问题描述
我正在对这样的列表使用prange进行迭代:
I am iterating using prange over a list like this:
from cython.parallel import prange, threadid
cdef int tid
cdef CythonElement tEl
cdef int a, b, c
# elList: python list of CythonElement instances is passed via function call
for n in prange(nElements, schedule='dynamic', nogil=True):
with gil:
tEl = elList[n]
tid = threadid()
a = tEl.a
b = tEl.b
c = tEl.c
print("thread {:} elnumber {:}".format(tid, tEl.elNumber))
#nothing is done here
with gil:
print("thread {:} elnumber {:}".format(tid, tEl.elNumber))
# some other computations based on a, b and c here ...
我期望这样的输出:
thread 0 elnumber 1
thread 1 elnumber 2
thread 2 elnumber 3
thread 3 elnumber 4
thread 0 elnumber 1
thread 1 elnumber 2
thread 2 elnumber 3
thread 3 elnumber 4
但是我得到了
thread 1 elnumber 1
thread 0 elnumber 3
thread 3 elnumber 2
thread 2 elnumber 4
thread 3 elnumber 4
thread 1 elnumber 2
thread 0 elnumber 4
thread 2 elnumber 4
那么,线程局部变量tEl为何会在线程之间被覆盖?我究竟做错了什么 ?谢谢!
So, somehow the thread local variable tEl becomes overwritten across the threads? What am i doing wrong ? Thank you!
推荐答案
Cython似乎故意选择从线程局部变量列表中排除任何Python变量(包括Cython cdef class
es). 代码
It looks like Cython deliberately chooses to exclude any Python variables (including Cython cdef class
es) from the list of thread-local variables. Code
我怀疑这是故意避免引用计数的问题-他们需要在循环结束时删除所有线程局部变量的引用计数(这不是一个无法解决的问题,但可能是一个很大的变化).因此,我认为它不太可能得到修复,但是文档更新可能会有所帮助.
I suspect this is deliberate to avoid reference counting issues - they'd need to drop the reference count of all the thread-local variables at the end of the loop (it wouldn't be an insurmountable problem, but might be a big change). Therefore I think it's unlikely to be fixed, but a documentation update might be helpful.
解决方案是将循环主体重构为一个函数,其中每个变量实际上都有效地局部"在该函数中,因此这不是问题:
The solution is to refactorise your loop body into a function, where every variable ends up effectively "local" to the function so that it isn't an issue:
cdef f(CythonElement tEl):
cdef int tid
with nogil:
tid = threadid()
with gil:
print("thread {:} elnumber {:}".format(tid, tEl.elNumber))
with gil:
print("thread {:} elnumber {:}".format(tid, tEl.elNumber))
# I've trimmed the function a bit for the sake of being testable
# then for the loop:
for n in prange(nElements, schedule='dynamic', nogil=True):
with gil:
f()
这篇关于Cython并行Prange-线程局部性?的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!