为什么不能腌制发电机? [英] Why can't generators be pickled?

问题描述

Python的泡菜(我在这里说的是标准的Python 2.5/2.6/2.7)不能泡菜锁，文件对象等.

Python's pickle (I'm talking standard Python 2.5/2.6/2.7 here) cannot pickle locks, file objects etc.

它也不能腌制生成器和lambda表达式(或任何其他匿名代码)，因为腌制实际上仅存储名称引用.

It also cannot pickle generators and lambda expressions (or any other anonymous code), because the pickle really only stores name references.

对于锁和与操作系统相关的功能，为什么您无法腌制它们的原因很明显并且很有意义.

In case of locks and OS-dependent features, the reason why you cannot pickle them is obvious and makes sense.

但是您为什么不能腌制发电机?

注意:为清楚起见-我对根本原因(或用于该设计决策的假设和选择)感兴趣，为什么，而不是因为它会给您泡菜错误".

Note: just for clarity -- I'm interested in the fundamental reason (or assumptions and choices that went into that design decision) why, not in "because it gives you a Pickle error".

我意识到这个问题有点宽泛，因此，以下经验法则是您是否回答的:如果提出这些假设，或者允许的发生器类型受到更多限制，酸洗发生器会再次起作用吗?"

I realize the question's a bit wide-aimed, so here's a rule of thumb of whether your answered it: "If these assumptions were raised, or the type of allowed generator somehow more restricted, would pickling generators work again?"

推荐答案

有很多与此有关的信息.有关该问题的正式用语"，请阅读(已关闭)Python bugtracker问题.

There is lots of information about this available. For the "official word" on the issue, read the (closed) Python bugtracker issue.

由于生成器本质上是一个强大的功能，因此我们需要保存其字节码，这不能保证在Python版本及其框架之间向后兼容，该框架保留了生成器的状态(例如局部变量) ，闭包和指令指针.后者相当麻烦，因为它基本上需要使整个解释器可腌制.因此，对酸洗生成器的任何支持都将需要对CPython的核心进行大量更改.

Since a generator is essentially a souped-up function, we would need to save its bytecode, which is not guarantee to be backward-compatible between Python’s versions, and its frame, which holds the state of the generator such as local variables, closures and the instruction pointer. And this latter is rather cumbersome to accomplish, since it basically requires to make the whole interpreter picklable. So, any support for pickling generators would require a large number of changes to CPython’s core.

现在，如果生成器的局部变量中出现了不被pickle支持的对象(例如，文件句柄，套接字，数据库连接等)，则该生成器将无法自动进行pickle，无论是否支持pickle我们可能实现的生成器.因此，在那种情况下，您仍然需要提供自定义的__getstate__和__setstate__方法.此问题使对生成器的任何酸洗支持都受到限制.

Now if an object unsupported by pickle (e.g., a file handle, a socket, a database connection, etc) occurs in the local variables of a generator, then that generator could not be pickled automatically, regardless of any pickle support for generators we might implement. So in that case, you would still need to provide custom __getstate__ and __setstate__ methods. This problem renders any pickling support for generators rather limited.

提到了两个建议的解决方法:

And two suggested workarounds are mentioned:

无论如何，如果您需要这样的功能，请查看可完成上述所有操作的Stackless Python.而且，由于Stackless的解释器是可腌制的，因此您还可以免费进行流程迁移.这意味着您可以中断小任务(Stackless的绿色线程的名称)，对其进行腌制，将其发送到另一台机器，对其进行腌制，恢复任务集，然后查看刚刚迁移的进程.这真是太酷了！

Anyway, if you need for a such feature, then look into Stackless Python which does all the above. And since Stackless’s interpreter is picklable, you also get process migration for free. This means you can interrupt a tasklet (the name for Stackless’s green threads), pickle it, send the pickle to a another machine, unpickle it, resume the tasklet, and voilà you’ve just migrated a process. This is freaking cool feature!

但是，以我的拙见，将这个生成器重写为简单的迭代器(即使用__next__方法的生成器)是解决此问题的最佳方法.因为迭代器的状态是明确的，所以它们在空间上很容易进行腌制.但是，您仍然需要处理显式表示某些外部状态的对象.你无法解决这个问题.

But in my humble opinion, the best solution to this problem to the rewrite the generators as simple iterators (i.e., one with a __next__ method). Iterators are easy and efficient space-wise to pickle because their state is explicit. You would still need to handle objects representing some external state explicitly however; you cannot get around this.

这篇关于为什么不能腌制发电机?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！