multiprocessing.Queue管道错误 [英] Broken pipe error with multiprocessing.Queue

问题描述

在python2.7中，当从函数内部进行初始化时，multiprocessing.Queue抛出错误错误.我提供了一个重现此问题的最小示例.

In python2.7, multiprocessing.Queue throws a broken error when initialized from inside a function. I am providing a minimal example that reproduces the problem.

#!/usr/bin/python
# -*- coding: utf-8 -*-

import multiprocessing

def main():
    q = multiprocessing.Queue()
    for i in range(10):
        q.put(i)

if __name__ == "__main__":
    main()

引发以下断管错误

Traceback (most recent call last):
File "/usr/lib64/python2.7/multiprocessing/queues.py", line 268, in _feed
send(obj)
IOError: [Errno 32] Broken pipe

Process finished with exit code 0

我无法解释原因.无法从函数内部填充Queue对象肯定很奇怪.

I am unable to decipher why. It would certainly be strange that we cannot populate Queue objects from inside a function.

推荐答案

在这里发生的是，当您调用main()时，它会创建Queue，将10个对象放入其中并结束该函数，并垃圾回收所有其内部变量和对象，包括Queue. 但是您会收到此错误，因为您仍在尝试发送Queue中的最后一个号码.

What happens here is that when you call main(), it creates the Queue, put 10 objects in it and ends the function, garbage collecting all of its inside variables and objects, including the Queue. BUT you get this error because you are still trying to send the last number in the Queue.

从文档文档:

"当进程首先将项目放入队列时，供料器线程即为 开始，它将对象从缓冲区转移到管道中."

"When a process first puts an item on the queue a feeder thread is started which transfers objects from a buffer into the pipe."

由于put()是在另一个线程中创建的，因此它不会阻止脚本的执行，并允许在完成Queue操作之前结束main()函数.

As the put() is made in another Thread, it is not blocking the execution of the script, and allows to ends the main() function before completing the Queue operations.

尝试一下:

#!/usr/bin/python
# -*- coding: utf-8 -*-

import multiprocessing
import time
def main():
    q = multiprocessing.Queue()
    for i in range(10):
        print i
        q.put(i)
    time.sleep(0.1) # Just enough to let the Queue finish

if __name__ == "__main__":
    main()

在将对象放入Queue中之前，应该有一种方法来执行join队列或块执行，您应该查看文档.

There should be a way to join the Queue or block execution until the object is put in the Queue, you should take a look in the documentation.

这篇关于multiprocessing.Queue管道错误的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！