在Python进程之间共享复杂的对象? [英] Sharing a complex object between Python processes?

问题描述

我有一个相当复杂的Python对象，需要在多个进程之间共享.我使用multiprocessing.Process启动这些进程.当我在其中共享multiprocessing.Queue和multiprocessing.Pipe的对象时，它们共享得很好.但是，当我尝试与其他非多处理模块对象共享一个对象时，似乎Python会分叉这些对象.是真的吗?

I have a fairly complex Python object that I need to share between multiple processes. I launch these processes using multiprocessing.Process. When I share an object with multiprocessing.Queue and multiprocessing.Pipe in it, they are shared just fine. But when I try to share an object with other non-multiprocessing-module objects, it seems like Python forks these objects. Is that true?

我尝试使用multiprocessing.Value.但是我不确定应该是哪种类型?我的对象类称为MyClass.但是当我尝试multiprocess.Value(MyClass, instance)时，它失败并显示:

I tried using multiprocessing.Value. But I'm not sure what the type should be? My object class is called MyClass. But when I try multiprocess.Value(MyClass, instance), it fails with:

TypeError: this type has no size

知道发生了什么事吗?

推荐答案

您可以使用Python的Multiprocessing"Manager"类和您定义的代理类来执行此操作.从Python文档中: http://docs.python.org/library/multiprocessing.html#proxy-objects

You can do this using Python's Multiprocessing "Manager" classes and a proxy class that you define. From the Python docs: http://docs.python.org/library/multiprocessing.html#proxy-objects

您要做的是为您的自定义对象定义一个代理类，然后使用远程管理器"共享该对象–在远程管理器"的链接文档页面中查看示例，其中文档显示了该示例如何共享远程队列.您将做同样的事情，但是对your_manager_instance.register()的调用将在其参数列表中包含您的自定义代理类.

What you want to do is define a proxy class for your custom object, and then share the object using a "Remote Manager" -- look at the examples in the same linked doc page for "remote manager" where the docs show how to share a remote queue. You're going to be doing the same thing, but your call to your_manager_instance.register() will include your custom proxy class in its argument list.

通过这种方式，您正在设置服务器以与自定义代理共享自定义对象.您的客户端需要访问服务器(同样，请参见出色的文档示例，其中介绍了如何设置客户端/服务器对远程队列的访问，但是您不是共享队列，而是共享对特定类的访问权限.)

In this manner, you're setting up a server to share the custom object with a custom proxy. Your clients need access to the server (again, see the excellent documentation examples of how to setup client/server access to a remote queue, but instead of sharing a queue, you are sharing access to your specific class).

这篇关于在Python进程之间共享复杂的对象?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！