腌制类定义 [英] Pickling a class definition

问题描述

有没有办法腌制类定义?

Is there a way to pickle a class definition?

我想做的是腌制定义(可以动态创建)，然后通过TCP连接将其发送，以便可以在另一端创建实例.

What I'd like to do is pickle the definition (which may created dynamically), and then send it over a TCP connection so that an instance can be created on the other end.

我知道可能存在依赖关系，例如该类所依赖的模块和全局变量.我也想在腌制过程中将它们捆绑在一起，但是我不关心自动检测依赖关系，因为如果用户有责任指定它们就可以了.

I understand that there may be dependencies, like modules and global variables that the class relies on. I'd like to bundle these in the pickling process as well, but I'm not concerned about automatically detecting the dependencies because it's okay if the onus is on the user to specify them.

推荐答案

如果使用dill，则可以将__main__视作一个python模块(大部分情况下).因此，您可以序列化交互式定义的类等. dill(默认情况下)也可以将类定义作为pickle的一部分进行传输.

If you use dill, it enables you to treat __main__ as if it were a python module (for the most part). Hence, you can serialize interactively defined classes, and the like. dill also (by default) can transport the class definition as part of the pickle.

>>> class MyTest(object):
...   def foo(self, x):
...     return self.x * x
...   x = 4
... 
>>> f = MyTest() 
>>> import dill
>>>
>>> with open('test.pkl', 'wb') as s:
...   dill.dump(f, s)
... 
>>> 

然后关闭解释器，并通过TCP发送文件test.pkl.现在，您可以在远程计算机上获取类实例.

Then shut down the interpreter, and send the file test.pkl over TCP. On your remote machine, now you can get the class instance.

Python 2.7.9 (default, Dec 11 2014, 01:21:43) 
[GCC 4.2.1 Compatible Apple Clang 4.1 ((tags/Apple/clang-421.11.66))] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import dill
>>> with open('test.pkl', 'rb') as s:
...   f = dill.load(s)
... 
>>> f
<__main__.MyTest object at 0x1069348d0>
>>> f.x
4
>>> f.foo(2)
8
>>>             

但是如何获取类定义?因此，这并不是您想要的.但是，下面是.

But how to get the class definition? So this is not exactly what you wanted. The following is, however.

>>> class MyTest2(object):
...   def bar(self, x):
...     return x*x + self.x
...   x = 1
... 
>>> import dill
>>> with open('test2.pkl', 'wb') as s:
...   dill.dump(MyTest2, s)
... 
>>>

然后在发送文件后……您可以获得类定义.

Then after sending the file… you can get the class definition.

Python 2.7.9 (default, Dec 11 2014, 01:21:43) 
[GCC 4.2.1 Compatible Apple Clang 4.1 ((tags/Apple/clang-421.11.66))] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import dill
>>> with open('test2.pkl', 'rb') as s:
...   MyTest2 = dill.load(s)
... 
>>> print dill.source.getsource(MyTest2)
class MyTest2(object):
  def bar(self, x):
    return x*x + self.x
  x = 1

>>> f = MyTest2()
>>> f.x
1
>>> f.bar(4)
17

因此，在dill中有一个dill.source，它具有可以检测函数和类的依赖关系的方法，并且可以将它们与pickle一起使用(大部分情况下).

So, within dill, there's dill.source, and that has methods that can detect dependencies of functions and classes, and take them along with the pickle (for the most part).

>>> def foo(x):
...   return x*x
... 
>>> class Bar(object):
...   def zap(self, x):
...     return foo(x) * self.x
...   x = 3
... 
>>> print dill.source.importable(Bar.zap, source=True)
def foo(x):
  return x*x
def zap(self, x):
  return foo(x) * self.x

因此，这并不是完美的"(或者可能不是预期的)……但是它确实对动态构建的方法及其依赖项的代码进行了序列化.您只是不了解该课程的其余部分，但是在这种情况下，不需要该课程的其余部分.

So that's not "perfect" (or maybe not what's expected)… but it does serialize the code for a dynamically built method and it's dependencies. You just don't get the rest of the class -- but the rest of the class is not needed in this case.

如果您想获取所有内容，则可以对整个会话进行腌制.

If you wanted to get everything, you could just pickle the entire session.

>>> import dill
>>> def foo(x):
...   return x*x
... 
>>> class Blah(object):
...   def bar(self, x):
...     self.x = (lambda x:foo(x)+self.x)(x)
...   x = 2
... 
>>> b = Blah()
>>> b.x
2
>>> b.bar(3)
>>> b.x
11
>>> dill.dump_session('foo.pkl')
>>> 

然后在远程计算机上...

Then on the remote machine...

Python 2.7.9 (default, Dec 11 2014, 01:21:43) 
[GCC 4.2.1 Compatible Apple Clang 4.1 ((tags/Apple/clang-421.11.66))] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import dill
>>> dill.load_session('foo.pkl')
>>> b.x
11
>>> b.bar(2)
>>> b.x
15
>>> foo(3)
9

最后，如果要透明地完成"传输，则可以使用pathos.pp或ppft，它们提供了将对象运送到第二个python服务器(在远程计算机上)或python的功能.过程.他们在引擎盖下使用dill，只是将代码通过电线传递.

Lastly, if you want the transport to be "done" for you transparently, you could use pathos.pp or ppft, which provide the ability to ship objects to a second python server (on a remote machine) or python process. They use dill under the hood, and just pass the code across the wire.

>>> class More(object):
...   def squared(self, x):
...     return x*x
... 
>>> import pathos
>>> 
>>> p = pathos.pp.ParallelPythonPool(servers=('localhost,1234',))
>>> 
>>> m = More()
>>> p.map(m.squared, range(5))
[0, 1, 4, 9, 16]

servers参数是可选的，这里只是连接到端口1234上的本地计算机…但是，如果您改为(或同时)使用远程计算机名称和端口，则会触发到远程机器-毫不费力".

The servers argument is optional, and here is just connecting to the local machine on port 1234… but if you use the remote machine name and port instead (or as well), you'll fire off to the remote machine -- "effortlessly".

在此处获取dill，pathos和ppft: https://github.com/uqfoundation

这篇关于腌制类定义的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！