Python functools lru_cache 与类方法:释放对象 [英] Python functools lru_cache with class methods: release object

问题描述

如何在类中使用 functools 的 lru_cache 而不泄漏内存?在下面的最小示例中，foo 实例不会被释放，尽管超出了范围并且没有引用(除了 lru_cache).

How can I use functools' lru_cache inside classes without leaking memory? In the following minimal example the foo instance won't be released although going out of scope and having no referrer (other than the lru_cache).

from functools import lru_cache
class BigClass:
    pass
class Foo:
    def __init__(self):
        self.big = BigClass()
    @lru_cache(maxsize=16)
    def cached_method(self, x):
        return x + 5

def fun():
    foo = Foo()
    print(foo.cached_method(10))
    print(foo.cached_method(10)) # use cache
    return 'something'

fun()

但是 foo 和 foo.big(一个 BigClass)仍然存在

But foo and hence foo.big (a BigClass) are still alive

import gc; gc.collect()  # collect garbage
len([obj for obj in gc.get_objects() if isinstance(obj, Foo)]) # is 1

这意味着 Foo/BigClass 实例仍然驻留在内存中.即使删除 Foo (del Foo) 也不会释放它们.

That means that Foo/BigClass instances are still residing in memory. Even deleting Foo (del Foo) will not release them.

为什么 lru_cache 会保留实例?缓存不使用一些哈希而不是实际对象吗?

Why is lru_cache holding on to the instance at all? Doesn't the cache use some hash and not the actual object?

在类中使用 lru_caches 的推荐方式是什么?

What is the recommended way use lru_caches inside classes?

我知道有两种解决方法:使用每个实例缓存或制作缓存忽略对象(不过这可能会导致错误的结果)

I know of two workarounds: Use per instance caches or make the cache ignore object (which might lead to wrong results, though)

推荐答案

这不是最干净的解决方案，但它对程序员来说是完全透明的:

This is not the cleanest solution, but it's entirely transparent to the programmer:

import functools
import weakref

def memoized_method(*lru_args, **lru_kwargs):
    def decorator(func):
        @functools.wraps(func)
        def wrapped_func(self, *args, **kwargs):
            # We're storing the wrapped method inside the instance. If we had
            # a strong reference to self the instance would never die.
            self_weak = weakref.ref(self)
            @functools.wraps(func)
            @functools.lru_cache(*lru_args, **lru_kwargs)
            def cached_method(*args, **kwargs):
                return func(self_weak(), *args, **kwargs)
            setattr(self, func.__name__, cached_method)
            return cached_method(*args, **kwargs)
        return wrapped_func
    return decorator

它采用与 lru_cache 完全相同的参数，并且工作方式完全相同.但是它从不将 self 传递给 lru_cache 而是使用每个实例的 lru_cache.

It takes the exact same parameters as lru_cache, and works exactly the same. However it never passes self to lru_cache and instead uses a per-instance lru_cache.

这篇关于Python functools lru_cache 与类方法:释放对象的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！