记忆到磁盘-python-永久记忆 [英] memoize to disk - python - persistent memoization

问题描述

有没有一种方法可以将函数的输出记录到磁盘上?

Is there a way to memoize the output of a function to disk?

我有一个功能

def getHtmlOfUrl(url):
    ... # expensive computation

，并希望执行以下操作:

and would like to do something like:

def getHtmlMemoized(url) = memoizeToFile(getHtmlOfUrl, "file.dat")

然后调用getHtmlMemoized(url)，以便对每个URL只进行一次昂贵的计算.

and then call getHtmlMemoized(url), so as to do the expensive computation only once for each url.

推荐答案

Python提供了一种非常优雅的方法-装饰器.基本上，装饰器是一个包装另一个功能以提供其他功能而不更改功能源代码的功能.您的装饰器可以这样写:

Python offers a very elegant way to do this - decorators. Basically, a decorator is a function that wraps another function to provide additional functionality without changing the function source code. Your decorator can be written like this:

import json

def persist_to_file(file_name):

    def decorator(original_func):

        try:
            cache = json.load(open(file_name, 'r'))
        except (IOError, ValueError):
            cache = {}

        def new_func(param):
            if param not in cache:
                cache[param] = original_func(param)
                json.dump(cache, open(file_name, 'w'))
            return cache[param]

        return new_func

    return decorator

一旦了解了这一点，就可以使用@ -syntax装饰"该函数，然后就可以准备就绪了.

Once you've got that, 'decorate' the function using @-syntax and you're ready.

@persist_to_file('cache.dat')
def html_of_url(url):
    your function code...

请注意，此修饰器是有意简化的，可能不适用于所有情况，例如，当源函数接受或返回无法进行json序列化的数据时.

Note that this decorator is intentionally simplified and may not work for every situation, for example, when the source function accepts or returns data that cannot be json-serialized.

有关装饰器的更多信息:如何制作一系列的函数装饰器?

More on decorators: How to make a chain of function decorators?

这是使装饰器在退出时仅保存一次缓存的方法:

And here's how to make the decorator save the cache just once, at exit time:

import json, atexit

def persist_to_file(file_name):

    try:
        cache = json.load(open(file_name, 'r'))
    except (IOError, ValueError):
        cache = {}

    atexit.register(lambda: json.dump(cache, open(file_name, 'w')))

    def decorator(func):
        def new_func(param):
            if param not in cache:
                cache[param] = func(param)
            return cache[param]
        return new_func

    return decorator

这篇关于记忆到磁盘-python-永久记忆的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！