基于try/finally + yield的Python析构函数? [英] Python destructor basing on try/finally + yield?

问题描述

我一直在测试受此 http://docs.python启发的肮脏黑客. .org/2/library/contextlib.html . 主要思想是将try/final思想带到类级别，并获得可靠且简单的类析构函数.

I've been testing a dirty hack inspired by this http://docs.python.org/2/library/contextlib.html . The main idea is to bring try/finally idea onto class level and get reliable and simple class destructor.

class Foo():
  def __init__(self):
    self.__res_mgr__ = self.__acquire_resources__()
    self.__res_mgr__.next()

  def __acquire_resources__(self):
    try:
      # Acquire some resources here
      print "Initialize"
      self.f = 1
      yield
    finally:
      # Release the resources here
      print "Releasing Resources"
      self.f = 0

f = Foo()
print "testing resources"
print f.f

但是它总是给我:

Initialize
testing resources
1

，绝不要释放资源".我的希望基于:

and never "Releasing Resources". I'm basing my hope on:

从Python 2.5版开始，现在允许在yield语句中使用 try的try子句...最终构造.如果发电机不是 在完成之前恢复(通过达到零参考计数或 通过垃圾回收)，生成器迭代器的close()方法 将被调用，从而允许任何未决的finally子句执行. 源链接

As of Python version 2.5, the yield statement is now allowed in the try clause of a try ... finally construct. If the generator is not resumed before it is finalized (by reaching a zero reference count or by being garbage collected), the generator-iterator’s close() method will be called, allowing any pending finally clauses to execute. Source link

但是似乎当类成员与类一起被垃圾回收时，其引用计数不会减少，因此结果生成器close()，因此永远不会调用它.至于报价的第二部分

But it seems when the class member is being garbage collected together with the class their ref counts don't decrease, so as a result generators close() and thus finally is never called. As for the second part of the quote

或被垃圾收集"

"or by being garbage collected"

我只是不知道为什么它不是真的.有机会让这种乌托邦发挥作用吗? :)

I just don't know why it's not true. Any chance to make this utopia work? :)

顺便说一句，这可以在模块级别使用:

BTW this works on module level:

def f():
  try:
    print "ack"
    yield
  finally:
    print "release"

a = f()
a.next()
print "testing"

输出将达到我的预期:

ack
testing
release

注意:在我的任务中，我无法使用WITH管理器，因为我正在线程的end_callback内部释放资源(它将不在任何WITH中).因此，在某些情况下由于某种原因而不会调用回调的情况下，我想获得一个可靠的析构函数

NOTE: In my task I'm not able to use WITH manager because I'm releasing the resource inside end_callback of the thread (it will be out of any WITH). So I wanted to get a reliable destructor for cases when callback won't be called for some reason

推荐答案

您遇到的问题是由参考周期和生成器上定义的隐式__del__引起的(它非常隐式，

The problem you are having is caused by a reference cycle and an implicit __del__ defined on your generator (it's so implicit, CPython doesn't actually show __del__ when you introspect, because only the C level tp_del exists, no Python-visible __del__ is created). Basically, when a generator has a yield inside:

• 一个try块，或等效地
• 一个with块

• A try block, or equivalently
• A with block

它具有类似__del__的隐式实现.在Python 3.3和更早版本上，如果引用循环包含其类实现__del__的对象(从技术上讲，在CPython中具有tp_del)，除非手动破坏了该循环，否则循环垃圾收集器无法清除它，而是将其粘贴在gc.garbage(import gc以获取访问权限)中，因为它不知道必须先收集哪些对象(如果有的话)以很好地"清理.

it has an implicit __del__-like implementation. On Python 3.3 and earlier, if a reference cycle contains an object whose class implements __del__ (technically, has tp_del in CPython), unless the cycle is manually broken, the cyclic garbage collector cannot clean it up, and just sticks it in gc.garbage (import gc to gain access), because it doesn't know which objects (if any) must be collected first to clean up "nicely".

由于类的__acquire_resources__(self)包含对实例的self的引用，因此形成了一个引用循环:

Because your class's __acquire_resources__(self) contains a reference to the instance's self, you form a reference cycle:

self-> self.__res_mgr__(生成器对象)->生成器框架(引用包含的本地元素)-> self

self -> self.__res_mgr__ (generator object) -> generator frame (referencing locals which includes) -> self

由于该参考循环，并且生成器中包含try/finally(创建与__del__等价的tp_del)，因此该循环不可收集，并且您的finally块永远不会除非您手动推进self.__res_mgr__(会破坏整个目的)，否则该命令会被执行.

Because of this reference cycle, and the fact that the generator has a try/finally in it (creating tp_del equivalent to __del__), the cycle is uncollectable, and your finally block never gets executed unless you manually advance self.__res_mgr__ (which defeats the whole purpose).

您的实验碰巧会自动显示此问题，因为参考周期是隐式/自动的，但是任何偶然的参考周期(其中周期中的某个对象具有带有__del__的类)都会触发相同的问题，因此即使您只是做了:

You experiment happens to display this problem automatically because the reference cycle is implicit/automatic, but any accidental reference cycle where an object in the cycle has a class with __del__ will trigger the same problem, so even if you just did:

class Foo():
    def __init__(self):
        # Acquire some resources here
        print "Initialize"
        self.f = 1

    def __del__(self):
        # Release the resources here
        print "Releasing Resources"
        self.f = 0

如果可以想象到，涉及的资源"可能导致以Foo的实例进行的引用循环，那么您将遇到相同的问题.

if the "resources" involved could conceivably lead to a reference cycle with an instance of Foo, you'd have the same problem.

这里的解决方案是以下一项或两项:

The solution here is one or both of:

1. 将您的课程设为上下文管理器，以便用户提供信息确定性终结(通过使用with块)所必需的，并为with块不可行(通过其自身资源进行清理的另一个对象状态的一部分)提供显式清理方法(例如，close)管理).这也是在从未使用过引用计数语义的大多数非CPython解释器中提供确定性清除的唯一方法(因此，如果有的话，所有终结器都将被不确定性地调用)
2. 移至Python 3.4或更高版本，其中 PEP 442 可解决以下问题:不可回收的循环垃圾产生了问题(从技术上讲，仍然可以在CPython上产生这样的循环，但只能通过继续使用tp_del的第三方扩展来进行，而不是通过更新以使用tp_finalize插槽来允许正确清除循环垃圾).它仍然是不确定性的清除操作(如果存在参考循环，则您正在等待循环gc某个时间运行)，但是它是可能，在这种情况下，3.4版之前的此类循环垃圾无法彻底清理干净.

1. Make your class a context manager so users provide the information necessary for deterministic finalization (by using with blocks) as well as providing an explicit cleanup method (e.g. close) for when with blocks aren't feasible (part of another object's state that is cleaned up through its own resource management). This is also the only way to provide deterministic cleanup on most non-CPython interpreters where reference counting semantics have never been used (so all finalizers are called non-deterministically, if at all)
2. Move to Python 3.4 or higher, where PEP 442 resolves the issue with uncollectable cyclic garbage (it's technically still possible to produce such cycles on CPython, but only via third party extensions that continue to use tp_del instead of updating to use the tp_finalize slot that allows cyclic garbage to be cleaned properly). It's still non-deterministic cleanup (if a reference cycle exists, you're waiting on the cyclic gc to run, sometime), but it's possible, where pre-3.4, cyclic garbage of this sort could not be cleaned up at all.

这篇关于基于try/finally + yield的Python析构函数?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！