Python的多处理的过程通信 [英] Process communication of Python's Multiprocessing

问题描述

我已经了解了Python多进程的Pipes/Queues/Shared ctypes Objects/Managers，并且想将它们与Linux的匿名管道，命名管道，共享内存，套接字等进行比较.我现在有以下问题

I've learned about Python multiprocess's Pipes/Queues/Shared ctypes Objects/Managers, and I want to compare them with Linux's anonymous pipes, named pipes, shared memory, socket, and so on. I now have the following questions

• Python的多处理的管道和队列模块基于匿名管道.它提供命名管道吗?

• The pipes and queue modules of Python's multiprocessing are based on anonymous pipes. Does it provide named pipes?

Python multiprocessing.sharedctypes是否支持独立进程 沟通?我认为它仅支持父子流程或 兄弟般的流程沟通.

Does Python multiprocessing.sharedctypes support independent process communication? I think it only supports father and child process or brotherly process communication.

其中仅用于亲子鉴定或 兄弟情谊，可以在独立流程之间进行交流 或其他主机?

Which of them are only used in the process of paternity or brotherhood, which can be communicated between independent processes or different hosts?

它们各自的特征是什么，我应该如何选择它们?

What are their respective characteristics, how should I choose them?

先谢谢了.

推荐答案

您的问题涉及面很广，大多数答案都可以在multiprocessing模块文档中找到.

Your question is quite broad and most of the answers can be found in the multiprocessing module documentation.

以下是一个简短的答案.

Here follows a somewhat short answer.

1. 多处理侦听器和客户端允许选择命名管道作为传输介质.

2. 来自文档:

1. The multiprocessing Listeners and Clients allow to choose named pipes as transport medium.

2. From the documentation:

multiprocessing.sharedctypes模块提供了用于从共享内存中分配ctypes对象的功能，这些对象可以由子进程继承.

The multiprocessing.sharedctypes module provides functions for allocating ctypes objects from shared memory which can be inherited by child processes.

您不能在没有父/子关系的流程中使用multiprocessing.sharedctypes功能.

You cannot use multiprocessing.sharedctypes functionalities across processes which don't have parent/child relationship.

Python multiprocessing模块最初是通过threading API实现的.到那时，它在支持的功能上有所增长，但核心思想保持不变. multiprocessing模块旨在处理Python进程系列.对于任何其他用途，subprocess模块是一个更好的选择.

Python multiprocessing module was initially implemented over the threading APIs. By the time, it grew in features it supports but the core idea remains the same. The multiprocessing module is intended to deal with Python process families. For any other use, the subprocess module is a better option.

要在多个主机之间分配任务和作业，有更好的解决方案可以抽象化低级基础结构.您可以查看Python项目，例如 Celery 或 Apache Mesos .

For distribution of tasks and jobs across multiple hosts, there are far better solutions abstracting the low level infrastructure. You can take a look at Python projects such as Celery or Luigi or more complex infrastructures such as Apache Mesos.

这篇关于Python的多处理的过程通信的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！