生成器在线程之间共享 [英] generators shared among threads
问题描述
这似乎是一个难以通过测试来回答的问题，所以我是b $ b希望有人知道......假设我有以下
发电机，g：
def f（）
i = 0
而True：
收益率我
i + = 1
g = f（）
如果我把g传递到各种线程，我希望它们永远是
产生一个独特的价值，我会有竞争条件吗？也就是说，是否有可能cpython解释器会在增量之后和产量之前中断一个线程，然后恢复另一个线程到
产生第一个线程的值，或者在恢复第一个线程之前增加存储的i或两者，
？如果是这样的话，如果我只是设置g，我会得到不同的
行为：
g = itertools.count（）
如果这两个成语都会给我一个竞争条件，我怎么可以去预防这样的？b $ b？我想过使用threading.Lock，但是我确定我不想锁定收益声明。
谢谢，
Jess
< je ********* @ gmail .COM>写道：
这似乎是一个难以通过测试回答的问题，所以我希望有人会知道...假设我有以下
发电机，g：
def f（）
i = 0
而真：
产量我
i + = 1
g = f（）
如果我将g传递给各种线程并且我希望它们始终能够产生一个独特的价值，我会拥有吗？竞争条件？也就是说，它是
是的，在恢复第一个线程之前你会。
？如果是这样，如果我只是设置g，我会得到不同的行为吗？
g = itertools.count（）
我相信当前的实现你会得到幸运，但
并不能保证在实现中即使是一个小的
错误修复也会存在这样的运气。不要这样做。
如果这两个成语都会给我一个竞争条件，我怎么能去预防呢？我想过使用threading.Lock，但我确定我不想锁定yield语句。
Queue.Queue通常是组织线程之间合作的最佳方式。
使用相当小的最大大小创建Queue.Queue，单个
专用线程放置连续项将itertools.count放到它上面
（当队列变满时隐式阻塞和等待），以及任何其他线程可以调用队列中的get并获取一个唯一的项
（如果队列变空，则隐式等待一点，直到
专用线程等待并再次填充队列）。 [[你可以选择
可以继承Queue并覆盖hook-method _get，它总是在正确锁定且因此序列化的条件下调用
;但是那个
可能被认为是一个相当高级的任务，因为这样的子类化没有在参考库中记录，只有在队列的来源中才能记录。]。 br />
Alex
[je ********* @ gmail.com]def f（ ）
我= 0
而真：
产生我
我+ = 1
g = f（）
如果我通过g我希望他们总能获得一个独特的价值，我会有竞争条件吗？
是的。
生成器可以在线程之间共享，但是它们不能同时从两个线程恢复
。
你应该换行它在一个锁中，以确保一次只有一个线程
可以恢复生成器。
几年前阅读这个帖子大致相同主题。
建议添加到线程模块的生成器。
http://groups.google.com/group/comp。 ... ... ede21f7dd78f34
该线程中还包含Queue.Queue的实现，其中
从生成器提供值，并且不需要单独的
线程来生成值。
HTH，
-
alan kennedy
------------------------------------- -----------------
电子邮件alan： http://xhaus.com/contact/alan
感谢伟大的建议，Alex。这是一个似乎
工作的子类：
来自队列导入队列的
来自itertools导入计数
类reentrantQueue（队列）：
def _init（self，maxsize）：
self.maxsize = 0
self.queue = []＃所以我们不必重写put（）
self.counter = count（）
def _empty（self）：
返回False
def _get（自我）：
返回self.counter.next（）
def next（self ）：
返回self.get（）
def __iter __（自我）：
返回自我
hi,
This seems like a difficult question to answer through testing, so I''m
hoping that someone will just know... Suppose I have the following
generator, g:
def f()
i = 0
while True:
yield i
i += 1
g=f()
If I pass g around to various threads and I want them to always be
yielded a unique value, will I have a race condition? That is, is it
possible that the cpython interpreter would interrupt one thread after
the increment and before the yield, and then resume another thread to
yield the first thread''s value, or increment the stored i, or both,
before resuming the first thread? If so, would I get different
behavior if I just set g like:
g=itertools.count()
If both of these idioms will give me a race condition, how might I go
about preventing such? I thought about using threading.Lock, but I''m
sure that I don''t want to put a lock around the yield statement.
thanks,
Jess
<je*********@gmail.com> wrote:
hi,
This seems like a difficult question to answer through testing, so I''m
hoping that someone will just know... Suppose I have the following
generator, g:
def f()
i = 0
while True:
yield i
i += 1
g=f()
If I pass g around to various threads and I want them to always be
yielded a unique value, will I have a race condition? That is, is it
Yes, you will.
before resuming the first thread? If so, would I get different
behavior if I just set g like:
g=itertools.count()
I believe that in the current implementation you''d get "lucky", but
there is no guarantee that such luck would persist across even a minor
bugfix in the implementation. Don''t do it.
If both of these idioms will give me a race condition, how might I go
about preventing such? I thought about using threading.Lock, but I''m
sure that I don''t want to put a lock around the yield statement.
Queue.Queue is often the best way to organize cooperation among threads.
Make a Queue.Queue with a reasonably small maximum size, a single
dedicated thread that puts successive items of itertools.count onto it
(implicitly blocking and waiting when the queue gets full), and any
other thread can call get on the queue and obtain a unique item
(implicitly waiting a little bit if the queue ever gets empty, until the
dedicated thread waits and fills the queue again). [[Alternatively you
could subclass Queue and override the hook-method _get, which always
gets called in a properly locked and thus serialized condition; but that
may be considered a reasonably advanced task, since such subclassing
isn''t documented in the reference library, only in Queue''s sources]].
Alex
[je*********@gmail.com]def f()
i = 0
while True:
yield i
i += 1
g=f()
If I pass g around to various threads and I want them to always be
yielded a unique value, will I have a race condition?
Yes.
Generators can be shared between threads, but they cannot be resumed
from two threads at the same time.
You should wrap it in a lock to ensure that only one thread at a time
can resume the generator.
Read this thread from a couple of years back about the same topic.
Suggested generator to add to threading module.
http://groups.google.com/group/comp....ede21f7dd78f34
Also contained in that thread is an implementation of Queue.Queue which
supplies values from a generator, and which does not require a separate
thread to generate values.
HTH,
--
alan kennedy
------------------------------------------------------
email alan: http://xhaus.com/contact/alan
Thanks for the great advice, Alex. Here is a subclass that seems to
work:
from Queue import Queue
from itertools import count
class reentrantQueue(Queue):
def _init(self, maxsize):
self.maxsize = 0
self.queue = [] # so we don''t have to override put()
self.counter = count()
def _empty(self):
return False
def _get(self):
return self.counter.next()
def next(self):
return self.get()
def __iter__(self):
return self
这篇关于生成器在线程之间共享的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
