Python fork():将数据从子级传递到父级 [英] Python fork(): passing data from child to parent

问题描述

我有一个主要的Python进程，并且使用

I have a main Python process, and a bunch or workers created by the main process using os.fork().

我需要将大量且相当复杂的数据结构从工作人员传递回主流程.您会为此推荐哪些现有库?

I need to pass large and fairly involved data structures from the workers back to the main process. What existing libraries would you recommend for that?

数据结构是列表，字典， numpy数组，自定义类(可以调整)和上述内容的多层组合.

The data structures are a mix of lists, dictionaries, numpy arrays, custom classes (which I can tweak) and multi-layer combinations of the above.

应避免使用磁盘I/O.如果我还可以避免创建数据副本(例如通过使用某种共享内存解决方案)，那也将很好，但并不是硬约束.

Disk I/O should be avoided. If I could also avoid creating copies of the data -- for example by having some kind of shared-memory solution -- that would be nice too, but is not a hard constraint.

出于这个问题的目的，必须使用os.fork()或使用其包装将克隆主进程的地址空间的包装程序来创建工作程序.

For the purposes of this question, it is mandatory that the workers are created using os.fork(), or a wrapper thereof that would clone the master process's address space.

这仅需要在Linux上运行.

This only needs to work on Linux.

推荐答案

multiprocessing的队列实现有效.

q = multiprocessing.Queue()
if (os.fork() == 0):
    print(q.get())
else:
    q.put(5)
# outputs: 5

这篇关于Python fork():将数据从子级传递到父级的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！