将Python对象传递给另一个Python进程 [英] Passing Python object to another Python process

问题描述

假设我们有一个用Python编写的服务器应用程序.

Let say we have a server application written in Python.

也可以说这个主服务器进程在启动时又分叉了两个进程.

Let also say that this main server process forked two more processes at the startup.

服务器正在等待其客户端，当一个客户端决定要从客户端的套接字传递两个派生进程中的哪个时.

Server awaits its clients, and when one comes decides to which of two forked processes should pass the client's socket.

我不想每次来客户时都派生一个进程.我想拥有固定数量的服务器，但是只有一台主服务器接收连接，然后将其传递给处理需要的特定工作客户端的服务器.

I do not want to fork a process each time a client comes; I want to have fixed number of servers, but one main server that receives a connection, then pass it to a server that deals with a specific work client asked for.

这应该是DOS攻击防护，作业分离等.

This should be a DOS attack protection, job separation, etc. etc.

是否有任何技巧可以在启动的Python程序之间传递Python对象.

Is there any trick to pass a Python object between started Python programs.

一些共享内存还是类似的东西?

Some shared memory or something like that?

是否可以腌制套接字对象并将其通过IPC工作?

Would pickling the socket object and pushing it through IPC work?

推荐答案

是否可以腌制套接字对象并将其通过IPC工作?

Would pickling the socket object and pushing it through IPC work?

不.在该对象内部是内核套接字的文件描述符或句柄.这只是进程在进行系统调用时用来标识套接字的数字.

No. Inside that object is a file descriptor or handle to the kernel socket. It's just a number that the process uses to identify the socket when making system calls.

如果您腌制该Python套接字对象并将其发送给另一个进程，则该进程将使用未打开的套接字的句柄.或更糟糕的是，该句柄可能引用了另一个打开的文件.

If you pickle that Python socket object and send it to another process, that process will be using a handle for a socket it didn't open. Or worse, that handle may refer to a different open file.

(在Linux上)最有效的处理方式如下:

The most efficient way to handle this (on Linux) is like this:

• 主进程打开侦听套接字(例如TCP端口80)
• 主进程派生N个孩子，他们全部继承打开套接字
• 他们都呼叫accept()并阻止，等待新的连接
• 当新的客户端连接时，内核将选择具有该套接字句柄的进程之一来接受连接；其他人将继续等待

• Master process opens listening socket (e.g. TCP port 80)
• Master process forks N children who all inherit that open socket
• They all call accept() and block, waiting for a new connection
• When a new client connects, the kernel will select one of the processes with a handle to that socket to accept the connection; the others will continue to wait

这样，您可以让内核处理负载平衡.

This way, you let the kernel handle the load balancing.

如果您不希望出现这种情况，则可以使用 (在UNIX中)将打开的套接字传递给另一个进程.再次，这不仅仅是句柄；内核有效地将打开的套接字复制到进程的打开文件列表中.这种机制称为SCM_RIGHTS，您可以在此处查看示例(在C语言中): http://man7.org/tlpi/code/online/dist/sockets/scm_rights_send.c.html

If you don't want this behavior, there is a way (in UNIX) to pass an open socket to another process. Again, this is more than just the handle; the kernel effectively copies the open socket to your processs's open file list. This mechanism is known as SCM_RIGHTS, and you can see an example (in C) here: http://man7.org/tlpi/code/online/dist/sockets/scm_rights_send.c.html

否则，您的主进程将需要有效地 proxy 与子进程的连接，从而降低系统的效率.

Otherwise, your master process will need to effectively proxy the connection to the child processes, reducing thr efficiency of the system.

这篇关于将Python对象传递给另一个Python进程的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！