在两次Python脚本运行之间将持久变量保存在内存中 [英] Keep persistent variables in memory between runs of Python script

问题描述

是否可以将结果变量保存在内存中，所以不必每次运行脚本的开头都重新计算它? 每次我运行脚本时，我都会对数据集(从磁盘上读取)进行一长串(5-10秒)确切的操作. 这并不是什么大问题，因为我非常擅长使用交互式编辑器在两次运行之间调试代码.但是有时候交互功能只是不能减少它.

Is there any way of keeping a result variable in memory so I don't have to recalculate it each time I run the beginning of my script? I am doing a long (5-10 sec) series of the exact operations on a data set (which I am reading from disk) every time I run my script. This wouldn't be too much of a problem since I'm pretty good at using the interactive editor to debug my code in between runs; however sometimes the interactive capabilities just don't cut it.

我知道我可以将结果写到磁盘上的文件中，但是如果可能的话，我想避免这样做.这应该是一种解决方案，它在我第一次运行脚本时会生成一个变量，并将其保留在内存中，直到关闭外壳本身或明确告诉它退出为止.像这样:

I know I could write my results to a file on disk, but I'd like to avoid doing so if at all possible. This should be a solution which generates a variable the first time I run the script, and keeps it in memory until the shell itself is closed or until I explicitly tell it to fizzle out. Something like this:

# Check if variable already created this session
in_mem = var_in_memory() # Returns pointer to var, or False if not in memory yet
if not in_mem:
    # Read data set from disk
    with open('mydata', 'r') as in_handle:
        mytext = in_handle.read()
    # Extract relevant results from data set
    mydata = parse_data(mytext)
    result = initial_operations(mydata)
    in_mem = store_persistent(result)

我暗示 shelve 模块可能是我在这里寻找的，但是看起来就像要打开一个搁置变量一样，我必须为持久对象指定一个文件名，所以我不确定它是否正是我要的内容.

I've an inkling that the shelve module might be what I'm looking for here, but looks like in order to open a shelve variable I would have to specify a file name for the persistent object, and so I'm not sure if it's quite what I'm looking for.

关于搁置做我想做的任何提示吗?还有其他想法吗?

Any tips on getting shelve to do what I want it to do? Any alternative ideas?

推荐答案

您可以使用reload全局函数重新执行主脚本的代码，从而实现类似目的.您将需要编写一个包装器脚本，该脚本会导入您的主脚本，向其询问要缓存的变量，然后在包装器脚本的模块范围内缓存该副本，然后在需要时(当您在stdin上按ENTER或其他命令)，它调用reload(yourscriptmodule)，但这一次将缓存的对象传递给它，以便您的脚本可以绕过昂贵的计算.这是一个简单的例子.

You can achieve something like this using the reload global function to re-execute your main script's code. You will need to write a wrapper script that imports your main script, asks it for the variable it wants to cache, caches a copy of that within the wrapper script's module scope, and then when you want (when you hit ENTER on stdin or whatever), it calls reload(yourscriptmodule) but this time passes it the cached object such that yourscript can bypass the expensive computation. Here's a quick example.

wrapper.py

import sys
import mainscript

part1Cache = None
if __name__ == "__main__":
    while True:
        if not part1Cache:
            part1Cache = mainscript.part1()
        mainscript.part2(part1Cache)
        print "Press enter to re-run the script, CTRL-C to exit"
        sys.stdin.readline()
        reload(mainscript)

mainscript.py

def part1():
    print "part1 expensive computation running"
    return "This was expensive to compute"

def part2(value):
    print "part2 running with %s" % value

wrapper.py运行时，您可以编辑mainscript.py，将新代码添加到part2函数中，并能够针对预先计算的part1Cache运行新代码.

While wrapper.py is running, you can edit mainscript.py, add new code to the part2 function and be able to run your new code against the pre-computed part1Cache.

这篇关于在两次Python脚本运行之间将持久变量保存在内存中的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！