迭代 Python 文件对象线程安全吗? [英] Is iterating over a Python file object thread safe?
问题描述
在寻找这个问题的巧妙解决方案时 我想知道对 Python 文件对象进行迭代是否是线程安全的.
While searching for a neat solution to this problem I was wondering if iterating over a Python file object is thread safe or not.
from concurrent.futures import ThreadPoolExecutor
import sys, time
f = open("big_file")
def worker():
running = True
while running:
try:
line = next(f)
except StopIteration:
return
# process line
time.sleep(3)
sys.stdout.write(line + "\n")
no_workers = 4
with ThreadPoolExecutor(max_workers=no_workers) as e:
for _ in range(no_workers):
e.submit(worker)
f.close()
我的问题是,如果上面的例子是安全的,或者如果不是,有什么简单的方法可以获取线程安全的文件对象(对于逐行读取文件,不需要写入).
My question is, if the example above is safe, or if not, what is an easy way to get a thread-safe file object (for reading a file line by line, writing is not required).
推荐答案
不,文件 I/O 不是线程安全的.锁定是一种解决方案,但我认为让单个线程处理每个外部资源的选项效果更好.其他线程将工作请求发送到 Queue.Queue
实例上的专用线程(并提供他们自己的另一个队列,以防他们需要返回结果),专用线程大部分时间都在等待在该队列上的 .get
上,每当它收到请求时,它都会处理它,可能会返回结果.
No, file I/O is not thread safe. Locking is a solution, but I think the option of having a single thread deal with each external resource works better. Other threads send work requests to the dedicated thread on a Queue.Queue
instance (and provide another queue of their own in case they need result(s) back), the dedicated thread spends most of its time waiting on a .get
on that queue and whenever it gets a request it deals with it, possibly returning the results.
这篇关于迭代 Python 文件对象线程安全吗?的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!